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Parallel Image Pi'ocessing 
(A Definition) 

" 'Parallel Image Processing' is image process- 
ing where all points of an image are operated upon 
simultaneously . ' 1 
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INTRODUCTION 


In anticipation of the future need for high speed image processing both on 
the ground and aboard spacecraft, a confei’ence on PARALLEL IMAGE PROC- 
ESSING FOR EARTH OBSERVATION SYSTEMS, sponsored by the Goddard 
Space Flight Center, The Office of Aeronautics and Space Technology of NASA 
Headquarters and The Office of Applications of NASA Headquarters was held 
on March 2 and 3, 1972 at the Goddard Space Flight Center. The aim of the 
conference was to bring together those involved in optical and digital image data 
processing technology, and those in the earth resources field. 

The conference was attended by 297 people. This large number of attendees 
required the transfer of the conference from the Bldg. 3 auditorium to the 
large Bldg. 8 auditorium. 

Eighty-two Goddard employees attended. Fifteen non-Goddard NASA em- 
ployees were present including 6 from NASA Headquarters. Ames, Marshall, 
JPL, Manned, Langley, and Wallops were represented. Sixty-eight Government 
employees attended representing the Army, Air Force, Navy, USGS, NOAA, 
DOT, NSA, USDA, NBS, CIA, the Smithsonian Astrophysical Observatory and, 
surprisingly, the Bureau of Outdoor Recreation. 

Twenty universities and 30 different companies were represented. Eight 
foreign nationals were present. 

Dr. Clark greeted the attendees for Goddard, and R. D. Ginter, Head of 
the Technology Applications Office of OAST at NASA Headquarters, gave greet- 
ings on behalf of OAST. Unfortunately, Dr. De Noyer, of the Office of Appli- 
cations at Headquarters, was unable to give the keynote address. His place 
was ably taken by Pitt Thome, the Deputy Director for Flight Programs, Earth 
Observations Programs, of OA. 

Technical Session I consisted of review papers of optical and digital image 
processing techniques. The aim of the session was to acquaint Earth Resources 
specialists with advanced methods of image processing. Session H in the after- 
noon turned the tables and acquainted the data processing experts with the 
problems of earth observation, including Goddard's ERTS ground processing 
requirements . 

Session III on Friday morning presented new developments in the image 
processing field. Real time devices to convert from the non-coherent to co- 
herent realm were-,described. Real time methods of performing cross 



correlation were described. One of the most interesting aspects of this session 
was the interleaving of Dr. Lee's, Dr. Strong's and Dr. Preston's papers. 
These three papers together appeared to point the way that optical techniques 
may enter the traditional computer field (as opposed to the strictly image proc- 
essing field). The session ended with excellent reviews of Marshall and JPL 
image processing work by John Williams of Marshall and Fred Billingsley of 
JPL. 

Friday afternoon four workshops were convened. All the workshops were 
well attended, and all created considerable excitement among the participants. 
Two of the workshops felt that permanent committees should be formed to 
address specific questions. 

The conference was a success due to the efforts of persons throughout 
NASA. 

Special mention is directed to Mike Michalak who was in charge of all the 
detailed arrangements. Most importantly there would have been no conference 
without Dr. Husain-Abidi's unstinting efforts as the confei’ence organizer. 

Due to the efforts of people in all parts of NASA and Goddard, we were 
able to have a conference that brought together many parts of the Government, 
industry, and the academic area. It is our hope that this conference mid these 
conference proceedings will not only benefit NASA but will also benefit a much 
larger image handling community throughout the Nation. 


David II. Schaefer 
Conference Chairman 
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WELCOMING REMARKS 


Dr. John F. Clark 

Director, Goddard Space Flight Center 


I don't know how we managed to have such beautiful weather today, but I 
am grateful for it. My meteorologist friends tell me that we can expect some 
thunderstorms this afternoon. It is not exactly conventional for March 2nd. 

It was both a surprise and a delight to us to have such a large turnout for 
this conference. As a matter of fact in the last couple of days we have shifted 
the site to this larger auditorium. 

I think it is fair to say, for other than the important series of Nimbus 
research and development spacecraft, that NASA spacecraft have historically 
been numbers collectors, as opposed to pictures. That is to say, they have 
measured quantities such as particle counts and magnetic field intensities. In 
a word, we have been dealing in the field of one-dimensional measurements. 

In contrast to this, the NASA Earth Observations Program of the future 
will be dealing predominantly with a collection of two-dimensional images, or 
one might say, measuring corn fields rather than magnetic fields. 

Our number collector spacecraft routinely condition these numbers with 
a large collection of well-developed devices; differentiators, counters, inte- 
grators, logarithmic amplifiers, and we have actually started recently to fly 
reprogrammable computers aboard our spacecraft. 

Image collectors presently have no such stable of devices which are 
especially designed for and suited to image inputs, much less computer sys- 
tems uniquely suited to process image data. 

Goddard is pursuing the development of image data processing systems, 
and in this we feel that an exchange of views is needed as the types of sub- 
systems that should be developed. This is a prime reason why this conference 
has been convened. 

Earth resource experts must know the potential of advanced image data 
systems, and conversely, the data processing community must become aware 
of the needs of the earth resource community. It is our hope that this con- 
ference will provide a large degree of cross-fertilization between these two 
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very important groups represented by you who are attending this conference. 

I hope that we can maximize the important signal- to- noise ratio in this process. 

I wish you success today and tomorrow. I sincerely wish I could spend 
most of it with you. 
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WELCOMING REMARKS 


R. D. Ginter, Director, Technology Applications Office 
Office of Aeronautics and Space Technology, NASA Headquarters 


My boss, Mr. Roy Jackson, the associate administrator of OAST, has 
asked me to express his personal interest and his very definite enthusiasm in 
your conference and in your efforts here. He had wanted to be here to make 
his own remarks, however, thp^e's a thing called authorization hearings for 
a subcommittee on the Hill this morning and that seemed to take precedence on 
his schedule. 

As some of you know, we in OART— and you will notice that I have been 
there a long time and I can't say OAST all the time— have been rather directly 
involved in much of your work, and because of this involvement are most 
interested, but not only in parallel image processing and what it can do for 
earth observations from spacecraft, but how this work and these techniques 
might be applied to the processing of information in many areas. 

As easily can be imagined from our title, our organization is very con- 
cerned about aeronautics, and there is a critical need for processing images 
quickly and accurately. These might allow planning systems to accommodate 
more airplanes per unit time, provide practical collision avoidance techniques, 
and maybe even to improve air traffic control procedures. 

The advent of the space shuttle, which has in fact brought about a sort of 
marriage between aeronautical and space technology, also brings with it 
similar problems. 

This spacecraft, really airplane, must obviously perform and operate as 
a high-performance airplane if it is to be successful, and it probably will not 
be allowed the luxury of waiting for ground control to decide whether it can 
accept another landing regardless of weather or other conditions. 

As most good bureaucrats do, you put in a bit of a plug for your own 
organization, mine is called the Office of Technology Applications, and we ai’e 
responsible in OAST for two primary activities. 

First, and probably most important, is the technology support to the 
Office of Applications, and to do this in a manner which is similar to what we 
have provided to the Office of Manned Space Flight on the shuttle and some of 
their other mission considerations. 
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We are now in the process of developing the programmatic and personal 
interface which should lead to a better understanding of their technology 
requirements and eventually implementation of the necessary work through our 
appropriate discipline divisions and in conjunction with the program divisions 
of the Office of Applications. 

The other major responsibility of my office is to manage an effort which 
uses the NASA capability to develop solutions in public sector problems. And 
that is a fairly broad charter and I will not attempt to explain all the wonder- 
ful little things we have going on in there. 

Maybe the most important thing though, is that both of these activities 
have some rather similar characteristics. One, is the need for understanding 
of what the actual problem is, development of technical managerial planning, 
and eventual demonstration of a solution. 

I have to admit, as an image processing, data gathering, data handling, lay- 
man, that lO 1 ^ bits seems somewhat awesome, and having worked on space 
station and others, have learned to parrot that number with great ease as 
though I knew what I was talking about. 

Certainly one of our OAST problems, and I suspect one of yours, is to 
develop the techniques which may most effectively or efficiently process the 
data and worry about how they are to be used. And this leads to a determina- 
tion of what data are actually needed, and I know of no easy or pat answer to 
that. 

So to me, the critical problem is to obtain and deliver, and I suspect with 
supreme confidence in the quality, those particles of intelligence which are in 
fact needed by the user. This problem is recurrent throughout most of the 
OAST activities as viewed from that standpoint. Whether we are concerned 
with technical support to the Office of Applications, helping the law enforcement 
organizations, or whether we are attempting to foster the development of a 
Goddard technique for quick automated detection of bacteria, the fundamental 
problem remains. We have to learn how to sense, process and finally deliver 
intelligence. 

OAST and myself wish you all good luck in your meeting and hope you can 
move significantly along this path. 
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KEYNOTE ADDRESS 


Pitt Thome, Deputy Director for Flight Programs, 

Earth Observations Programs, Office of Applications, NASA Headquarters 


Ladies and gentlemen, Dr. DeNoyer sends his regrets he cannot be with 
you here today and give this address, but he asked me to stand in for him. I 
think you can understand that the hearings going on in defense of our budget 
next year is rather basic in excusing him for being absent this morning. 

I would like to start off by outlining some results of a study that I recently 
had the opportunity to review. The study is about at the half-way mark, but I 
think some of the preliminary results of that study are very important and 
germane to this conference. 

The study dealt with the data management problems of the space station, 
in particular the plans for the data dissemination system for our space station: 
how to handle the space station experiment data; how to get the data to the users 
in a timely manner, and how to provide the proper amount of interactive con- 
trol that these users may require with the experiments being conducted on the 
space station. 

The time period of the space station that was postulated for the study is the 
late 1970s, early 1980s, and like all the space station studies they address all 
the relevant space science and applications disciplines. 

This latest study reaffirmed what has been pointed out by several earlier 
studies, that the prime driver in the whole data management area deals with the 
disciplines inherent in the earth observations programs, and particularly the 
earth resources survey disciplines. 

The study assumed that in the earth observations area there were basically 
six types of experiment areas that would be covered by the space station. These 
included weather and atmospheric studies, world-land use mapping, study of 
air and water pollution, the resource recognition and identification, natural 
disaster assessment, and ocean resource surveys. 

From these experiment areas the study identified the type of data that is 
required to do research in these areas, as to the characteristics of the data, 
that is the requirements for the quantity, the accuracy and the timeliness of 
the data. 



The study then went on and identified the sensors that will be required to 
collect this data. As an example, they came up with a candidate payload which 
included just for the earth observations areas, six different imagers, metric 
cameras, multispectral scanners, multispectral cameras, three radiometers, 
two spectrometers and a host of other miscellaneous types of sensors required 
to collect the required R&D data. These assumptions, as far as sensors and 
associated data requirements in this time frame, were arrived at by extrapo- 
lating from the l’equirements placed in the past on the aircraft program, the 
ERTS program and the Skylab program. 

These postulated future requirements maintain the trend that we have been 
seeing for several years, towards the need for ,more data, the need for data 
with higher resolution, both spectral and spatial, and the need for increased 
accuracy. 

The results of the study I found extremely interesting. I don't remember 
exactly the quantity of data returned, but my recollection was just for the earth 
resources area, that this type of space station that was postulated as needed 
in the late '70s, early '80s, would return up to 10^ bits per day and 50, 000 
images a day. 

If allowance is made for some reduction in these requirements, reducing 
some of the requirements on data collection and other rather simple techniques, 
the study concluded that in addition to the earth resources processing capability 
that we now have at the Manned Spacecraft Center to support the aircraft program, 
to support the Skylab program, in addition to the data processing facilities we 
have here at Goddard to support the ERTS program, and in addition to the 
capability we have at the Marshall Space Flight Center and Slidell Computer 
Facility, that 7 to 8 new centers would be required to process this data. 

Seven or 8 new centers in terms of data handling capacity comparable to what 
we have today here at Goddard would be required to process this data. 

If one were to also consider the resources required to analyze this data, 
the problem could be even more staggering. 

The reason I summarize these study results, is to illustrate the potential 
situation that we could get into, what has commonly been called and very aptly 
so, data pollution. If we are not careful, and if we do not do our homework 
well we may, indeed, be faced with a serious data pollution problem. 

We have the increased pressure, as I indicated before, to increase cover- 
age, to provide more data, to provide higher resolution both spectral and 
spatial, and to provide increased accuracy. If we yield to these pressures 
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indiscriminately, we will be collecting more and more data that must be sent 
back down to the earth, quantities that cannot be effectively used. 

This then puts strain on our bandwidth capacity in spacecraft, it puts 
strain on our computer capacity to process all this data, it puts strain on the 
analysis effort to effectively utilize this data and turn it into information. 

Hence, because of the strain in these various areas, it will be very difficult 
to even keep up with the data. Therefore, I think we must take the necessary 
steps to minimize data pollution as much as possible. 

Well, what are some of the possible solutions? I wanted to mention several 
areas that should be pursued, areas that are not new to anybody here, but ones 
that a conference of this nature on looking at imaging processing systems 
should keep in mind. We must not just arbitrarily embark on improving the 
techniques for processing larger and larger amounts of data indiscriminately, 
but explore ways of reducing the quantity of collected data. 

I think, first of all, we have to be selective on where the data needs to be 
collected. It should start right at the source of the problem, that is, the 
whole data collection phase. We should be selective on how often the data is 
collected. If the data is not needed but once every quarter, no sense in 
collecting it every week and archiving it or choking the processing facilities. 

We should be selective as to the spatial and spectral resolution required. It 
doesn't make sense to keep pushing for higher and higher resolution in those 
areas where it is not required, and to confine data collected that we do obtain 
for higher resolution, to those areas for which it is directly required. 

We have learned a lot about this in the last few years, with the data from 
the aircraft program, and we are going to leai’n a lot more in the next few 
years with the experiments that will be conducted on the ERTS program and the 
Skylab program. However, I think there are other things we need to give 
consideration to at this point in time. 

There ought to be more consideration given to the use of real time inter- 
action with the data collection phase to eliminate the collection of data that is 
not useful, rather than collecting the data for periods of weeks and months 
before it is given to the ultimate user and analyzed and determined whether it 
is useful. There must be a quick response time back into the data collection 
phase. We need to look more at onboard processing to reduce the amount of 
data that is transmitted back to the earth; give greater consideration to the 
use of both unsupervised and supervised programs in our spacecraft. We also 
need to look at the total system concepts to provide efficient data dissemination. 
Is the best approach one where the data from satellites is transmitted directly 
to a central receiving and processing station and then data disseminated at 
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the users? Should then a data relay satellite be used to relay this processed 
data out of the central receiving and processing center to the ultimate users ? 
Would that indeed improve the effectiveness in the dissemination process? 

Or with more onboard processing, is it possible and desirable to transmit the 
data directly to regional centers by means of a data relay satellite ? How can 
we provide for more real time interaction by the users to reduce the data 
collected ? 

Another area that needs studying, is how to make the most effective use of 
the unique capabilities of the space shuttle. This capability will present ad- 
ditional problems dealing with the timelier readout of data, the timelier uliti- 
zation of data and the interaction with the collection process. This needs to 
be studied. 

This conference cannot find solutions to all these problems. I think that 
we should first of all assess what technology we have now that could be used to 
solve these problems. We should also identify what new technology is needed 
to assist in minimizing the data pollution problem. I think to do this properly 
it has to be put in a framework that requires a systems look. Maybe all this 
cannot be accomplished here in this conference, but I think a start should be 
made on it. A system look at the data collection phase, minimizing the data 
collected, going more to onboard processing of the spacecraft — considering also 
new techniques in ground processing, and new techniques for the dissemination 
of data the to the ultimate user in a more effective manner. All system con- 
cepts of this nature and others should be considered. 

In conclusion, I would like to say that this conference is extremely timely. 
With the ERTS and Skylab capability that we will have shortly, the amount of 
data being generated by these satellites, the data pollution problem is, indeed, 
a very real problem. Solutions are urgently needed to build into the next gener- 
ation of spacecraft and ground systems, if we are to effectively realize the 
great potential inherent in our remote sensing technology. 



REVIEW OF COHERENT OPTICAL DATA PROCESSING 


A. Vander Lugt, Research Manager 
Radiation, Inc. , Ann Arbor, Michigan 



INTRODUCTION 

When we consider the vast amount of data collected for the study of Earth 
Resource imagery, we find that we have many users wanting to extract specific 
information from large data bases. As the keynote speaker indicated, the 
total problem is one of data collection, data storage, data processing, data 
dissemination, and data utilization. The principal topic of this conference is 
to discuss parallel processing of images collected by earth observation sys- 
tems. Parallel processing is clearly a requirement when one considers that 
a single image formed by a sensor in just one part of the spectral region may 
contain from 10 6 to 10 8 pixels. Furthermore, if we consider that many sen- 
sors may be acquiring data simultaneously, we find that the total amount of 
data collected per unit time is staggering. 

The key question to ask is how we can process this data to extract the 
information of interest to a particular user. The solution to this problem 
presupposes that the user knows what information he is trying to extract and 
that he has some knowledge of the sensor characteristics. Our efforts here 
ought to focus on determining what kinds of parallel processing systems are 
available for potential application to image processing. I shall review the 
fundamental features of coherent optical data processing systems and give 
some representative illustrations of how they can be used. 

Optical processing systems are inherently parallel processors. They are 
capable of performing operations on images containing millions of pixels in 
extremely short time intervals. The throughput is , in fact, generally limited 
by peripheral devices and not by the processing speed of the optical system 
itself. If, in addition, the optical system is coherently illuminated, we have 
the advantage of being able to operate on the two-dimensional Fourier trans- 
form of the image and can therefore perform two-dimensional spatial filter- 
ing operations in parallel. 

The basic theory of optical spatial filtering is well developed (Fig. 1). 

The data to be processed modulate a coherent beam of light. The light dif- 
fracted by the data is collected by a lens that displays the two-dimensional 
Fourier transform of the data at the image plane (frequency plane) of the 
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primary light source. A transparency or mask placed in the frequency plane 
is the spatial filter that modified the Fourier transform of the data. The 
filtered data, displayed at the image plane, can then be detected for further 
processing. 

Since the desired filtering operations frequently require the use of filters 
that have complex values (both phase and amplitude information) , means for 
x'ecording such filters are required. Spatial carrier frequency filters, in 
which a complex -valued function is encoded as a non-negative function, can be 
constructed by using a technique closely related to one used in holography for 
recording holograms. The filter construction system, therefore, consists of 
a signal beam in which the Fourier transform of the desired impulse response 
is displayed, and an off-axis reference beam. If the sum of the reference 
beam and the Fourier transform is detected by a square-law detector, the re- 
corded function can be used directly as the desired filter. Alternatively, the 
filter may be generated by a computer and recorded by one of several means. 

One application of coherent optical systems is immediately suggested if 
we note that the Fourier transform exists as a physical, measurable light 
distribution. Figure 2 shows an aerial image of the surface of an area of 
water. Within the small inserts we note the Fourier transform (which is an 
estimate of the power spectrum) of several subregions of the image. From 
the Fourier transform we can learn something about the structure or texture 
of the surface of the water. This same technique could be used to classify 
terrain in various parts of the spectrum according to their distinctive spatial 
frequency content. This could be the first step in many data reduction 
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processes. The technique shown here is only one of several that may be ap- 
plied, depending on the specific information to be extracted. 

More powerful data processing operations can be implemented, however, 
if we use spatial filters in the frequency plane to modify the Fourier trans- 
form of the image. In this paper we review two related applications which 
illustrate some features of parallel processing. Both are concerned with 
image correlation, that is, to solve the problem of locating one image (or a 
part of an image) relative to another. The two images do not necessarily have 
to be produced by the same sensor nor do they have to be taken at the same 
time or from the same vantage point. 

The first application could be termed image mapping. Suppose that we 
have a reference library of images obtained from a given sensor and that we 
construct spatial filters for these images. It may be noted that if we consider 
the optical processor as a computer, then the filters are equivalent to the 
memory of the computer. Each filter contains as much information as an 
entire image, e. g. , 10 6 to 10 8 data points. Several hundred filters can be 
recorded on one role of film to give a mass storage capability. Once the filter 
library has been recorded, we are in a position to process data. 

Figure 3 illustrates the steps required for image mapping. Figure 3a can 
be considered a frame of imagery which was recorded and stored as a spatial 
carrier frequency filter to form a part of a reference library. Figure 3d 
shows the autocorrelation of Figure 3a; its position serves to associate the 
output plane coordinates with the reference data. Figure 3e shows the cross- 
correlation of Figure 3a with the new image, Figure 3b. Note that the cross- 
correlation peak has moved upwards and to the right, indicating the position of 
the frame of imagery relative to the stored reference function. In this 
example we show that we do not need to use all of the imagery to locate its 
position; rather, only the portion shown in Figure 3c was used. The reduced 
area still provides a cross -correlation peak with sufficient signal-to-noise 
ratio to measure accurately the location of the entire frame. It is not, in 
this example, important which part of the imagery is used; every part of 
Figure 3b contained in Figure 3a correlates equally well, and each part pro- 
duces a cross -correlation peak having the same location so that the position 
of the mission imagery is unambiguous. The intensity of the cross -correlation 
peak did not vary as a function of which subportion was illuminated because 
the terrain has very small changes in elevation and In illumination; different 
camera viewing angles therefore caused only small changes in the geometry. 

Since it may be an expensive proposition to obtain reference imagery 
covering large areas of the earth, we have investigated an alternative means 
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for mapping. If we can find the center coordinates and orientation for at least 
one frame of imagery, and if there is some overlap of the frames, we can 
carry out the mapping by finding the position of each frame relative to the pre- 
vious frame. The key to this technique is the use of an erasible recording 
material in the frequency plane which will be described later. 

A related application is that of cloud motion analysis. Again, we have a 
sequency of frames of imagery and want to determine, in this case, the motion 
of the cloud patterns. From these movements we can make a first order esti- 
mate of the wind velocities. The procedure is similar to that described before 
except that now the correlation plane contains many subcorrelation peaks, one 
for each cloud pattern, that must be sorted. 

The motion of individual patterns can be determined by allowing only the 
pattern of interest to appear in an aperture placed in the input plane of the 
system. By moving the aperture in the input plane and recording the position 
of the cross -correlation peak in the output for each position of the aperture, a 
synoptic picture of the cloud motion over the entire field of view can be devel- 
oped. Cloud rotation can be measured simply by rotating one frame until the 
cross -correlation in the output plane is maximized. 

The circled areas shown in Figure 4 show the approximate locations of the 
various cloud regions that were sampled. By means of a movable aperture, 
which may be under operator control, we sort the subcorrelation peaks. Each 
division (1 inch) on the scales in Figure 5 represents approximately 25 mph. 
The variable geometry of the scene has not been considered in determining 
velocities. 

The magnitude of the cross -correlation is a measure of the similarity of 
the patterns in each frame. If the shape of the patterns changes significantly 
between frames, the correlation peak will be correspondingly reduced. Thus, 
this technique also offers an objective measure of the similarity between pat- 
terns which is useful in determining the rate at which clouds change shape. 

The same comments apply to measuring changes in, for example, the seasonal 
texture of terrain. 

At the Electro-Optics Center we have used photoplastic devices to con- 
struct spatial carrier -frequency filters in situ in real time. Spatial filters for 
use in applications such as pattern recognition, target detection, character 
recognition and signature detection have usually been recorded on a nonreus- 
able material such as photographic film. The required filters were, there- 
fore, recorded before the data was processed to form a reference library. 




Figure 4. Cloud Coverage 

This page is reproduced at the 
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In some applications it is desirable to construct a filter in situ for a pat- 
tern not contained in the library. Although this can be done by using self- 
developing materials such as photopolvmers, these materials are not reusable. 
Reusable materials such as photochromies are either insensitive and are not 
practical for use in spatial filters. The photoplastic realtime device developed 
at the Electro-Optics Center is reusable. It can be recycled many times; 
moreover, its sensitivity is comparable to that of 649F photographic emulsion 
on which filters are typically recorded. This device considerably extends the 
flexibility and usefulness of coherent data processing systems because it gives 
us the opportunity to update the memory of the computer. 

We have also made efforts to improve the flexibility of optical processing 
by developing a hybrid system in which the optical system performs the high- 
speed, fixed routine operations while a digital computer performs low-speed 
flexible routine operations. Furthermore, such a system has considerable 
potential for use in a human interactive system. Figure 6 shows a hybrid 
optical system which is a dual-purpose system, i.e. , it can both generate 



Figure 6. Hybrid Optical System 


This page is reproduced at the 
hack of the report In a different 
reproduction method to provide 
Iretter detail. 



filters and process data. Of particular interest is the readout system which 
consists of the image orthicon, a threshold device, a cell generator, a PDP- 
8/1 computer, a graphics displaj' unit, and a TV monitor. 

The TV camera detects the light distribution at the output plane of the 
optical system, the resulting video signal is thresholded and fed into a cell 
generator which is a high-speed, double-buffered memory composed of shift 
registers and various other logic circuits. The cell generator organizes the 
thresholded video signal into a number of cells; each cell represents a small 
rectangular area at the output plane of the optical system. A cell is active if 
there is at least one video peak exceeding threshold in the area represented 
by the cell. The state of a cell is indicated by the state (0 or 1) of the cor- 
responding flip-flop in the shift registers. The cell generator can be operated 
in an almost arbitrary format, but it normally creates a 32 x 32 array of cells. 

The function of the electronic readout system is to collect, collate, and 
display the processed data. In a typical processing application the output of 
the optical system contains correlation peaks which must be detected and proc- 
essed further. A basic concept in cross -correlation operations is that the out- 
put data rate can be reduced significantly relative to the input data rate. This 
concept is particularly important in optical processing because a frame of in- 
put data may contain 10® pixels. Since correlation is a linear operation, the 
output detector must have the same number of resolution elements as the input 
and, unless data reduction takes place, extremely high data rates may be 
required to transfer the processed data to the user in a useful form. For 
example, if a frame of imagery is processed in 1 sec, data rates in excess of 
100 Mb/s are required at the output. Clearly, such data nates cannot be 
handled by conventional data display systems. 

A third area in which further progress is needed is that of input interface 
devices. Although photographic film is an ideal input media, we often would 
like to process electrical analog or digital data. Data composing devices such 
as the light valve, ferroelectric materials, thermoplastic materials, and the 
like are useful for inputting two-dimensional data whereas acousto-optic cells 
have potential for inputting one -dimensional time signals. As further improve- 
ments are made in input and output interface devices, we can expect to see 
even wider use made of coherent processing systems. 

The illustrations given here by no means exhaust the list of applications. 
But spectral analysis and correlation offer two powerful operations that are of 
great general utility. For example, another important application is image 
deconvolution (or image enhancement or image deblurring). This method be- 
longs, however, to the correlation class of operations. Other applications are 
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differentiation, subtraction processes for change detection, and cross spectral 
estimation. Furthermore, the optical processor can perform nonlinear as 
well as linear operations. 

In the future, then, we can expect to see optical processors used in con- 
junction with digital computers to carry out the required processing operation. 
Each system will be used to perform those operations for which it is best 
suited; the digital computer for numerical calculations and for operations not 
easily implemented optically, and the optical computer for performing integral 
operations and other parallel operations on the entire image simultaneously. 
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OPTICAL PROCESSING IN INCOHERENT LIGHT 
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ABSTRACT N 7 3 ~ 18686 

Not every kind of optical input signal can be processed 
in a coherent optical processor. Light from self-luminous 
or incoherently illuminated objects has a randomly corru- 
gated light phase front. Consequently, the Fourier trans- 
form of the input signal does not appear in the transform 
plane. Therefore, before they can be used in coherent proc- 
essors, these objects must be converted into usable formats, 
such as non -diffusing transparencies. Unfortunately, no 
good, convenient, real-time method of preparing instant 
transparencies exists at this time. 

As an alternative to the coherent processor, one may 
choose to utilize the filtering properties of an incoherent 
imaging system. In such an imaging system, the input 
object is convolved with the image of a point-object. This 
point-object image, more often called the intensity impulse 
response or the point spread function of the system, can be 
controlled by proper design of the imaging system. In 
particular, holography provides an extremely convenient 
means for designing the impulse response. Although no 
Fourier plane exists in an incoherent system, proper 
tailoring of the impulse response allows implementation of 
many linear filtering operations including matched filtering 
and low-pass filtering. Two applications of this approach 
are optical filtering of oscilloscope traces and optical 
recognition of characters printed on ordinary paper. Un- 
fortunately, the requirement that the image be everywhere 
positive rules out high-pass filtering and hence precludes 
contrast enhancement. 

The basic principles of incoherent image processing 
are presented and the advantages and disadvantages of this 
approach are discussed. 
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INTRODUCTION 


As man' s ability to collect ever greater masses of data in ever shorter 
time intervals outstrips his ability to process these data, he is led to consider 
optical processors, drawn by the extremely large information handling capa- 
city of such systems. Although interest commonly centers on the coherent 
optical processor because of its great versatility and elegance, there are times 
when such a system is seriously slowed down because the data are not present 
in a usable format. To be specific, it is essential that the input to a coherent 
processor be in the form of an equivalent transparency which modulates the 
amplitude and phase of the light front with information but does not diffuse or 
destroy the coherence of the illumination. In particular, patterns imprinted on 
paper or real scenes viewed directly do not satisfy this non-diffusing criterion. 

An older class of processors known as incoherent is able to work directly 
from diffuse or incoherently illuminated inputs. The possibility of processing 
CRT patterns or objects viewed in filtered sunlight then arises. In this paper 
we discuss briefly the possibilities and shortcomings of these incoherent proc- 
essors. We shall confine our discussion to systems capable of operation in 
real time and to systems based on physical optics. We shall thus not deal 
with photographic processing or the well-established geometrical optics 
approach called optical correlography. 

COHERENT PROCESSORS 

Let us begin our discussion of incoherent processing techniques by looking 
at the more familiar coherent processor represented in Fig. 1. We note that, 
reduced to simplest terms, it is an imaging system. An object o(x 0 ) is 

located in the front focal plane of lens L-j ; the spatial Fourier transform O(o>) 
of this object is formed in the back focal plane. O(co) is then retransformed by 
lens L,, into the image i(x) in the output plane. The Fourier plane exists be- 
cause of the coherence of the illumination. One can modify the image by manipu- 
lating the amplitude transmittance in this Fourier plane, for example by insert- 
ing a transparency H(co). Because one can thus operate directly on the spectrum 
of o(x Q ), the implementation of filtering operations is straightforward and low- 
pass, hi-pass, band-pass, and matched filtering are all possible. With skill 
and imagination, differentiation, addition and subtraction can also be achieved. 
The image is the transform of the product 0(co)H(co) and is therefore given by 
the convolution of the transforms of o and h, i.e. , 

i(x) = /dcoO(co)H(co)e _iu>x (1) 
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= K 0 (X 0 )11(X- X 0 ) 


( 2 ) 


= o(x) * h(x) (3) 

where the asterisk is a shorthand notation for convolution. In all that follows, 
we shall usually represent spatial functions by lower case and the correspond- 
ing transforms by upper case letters with the notation 

f(x) <=> F(co) (4) 

When H(co) is different from unity, the images may no longer look at all 
like the object. In the case of matched 10 filtering (H(gj) = O*(oo)) , the "image" 
will in fact be the autocorrelation function of the object: 


i(x) = JdcoO(cj)0*(oj)t~ iLJX 

(5) 

= /dx 0 o(x+x 0 )o*(x o ) 

(6) 

= o(x)©o*(x) 

n 


where we introduce the notation© for the correlation operation. 

For all its flexibility and high data handling capacity, the coherent optical 
processor suffers from one severe defect. The object must be present in the 
form of a non-diffusing transparency. Thus, either a photographic trans- 
pai’ency must be prepared (thereby precluding real-time operation) or a 
coherent light gate must be employed. Although a wide range of candidates 
exists, there is as yet no satisfactory solution for high data rates. While work 
goes forward to develop such real-time transparencies, we shall turn our 
attention to another class of processors which are capable of operating directly 
from incoherent or diffusely illuminated sources. They do not have quite the 
flexibility of the coherent systems but do nevertheless have useful applications. 

INCOHERENT PROCESSORS 

General 

The incoherent optical processor is also based on the properties of an 
imaging system such as that shown in Fig. 2. Note that the limitation to unit 
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magnification is for simplicity only. 9 Although there is no plane in which the 
Fourier transform is displayed, one can nevertheless still modify the spatial 
frequency response of the imaging system to implement a number of useful 
filter functions. Note from Fig. 2 that we are now dealing with intensity objects 
|o(x Q )| 2 and intensity images |i(x)| 2 . We shall deal with spatial frequencies of 
these intensity distributions. 

If the system of Fig. 2 were ideal, the image would resemble the object in 
every detail. As we know, this perfection can never be attained. The image 
always exhibits less contrast and less definition than the object. In other words, 
the system is operating as a low-pass filter due to the presence of a finite 
aperture or pupil. Let us consider the simplest object, the point object shown 
in Fig. 3. The image of such an object is nqt a point but an Aii’y diffraction 
pattern, ^ characterizing the pupil. As the diameters of the imaging lens and 
pupil increase, the image approaches more and more closely the ideal point 
response. 


Object Image 

plane H (u>) plane 



f f f f 


i (x) = f o ( x 0 ) h ( x- x o ) dx 0 
h(x) < = > H (to) 

Figure 1. Coherent Optical Processor. The Image i(x) of the Object 
o(x Q ) is Processed by Modifying the Transmittance in the Spatial 

Frequency Plane 
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Figure 2. Incoherent Optical Processor. Processing is Accomplished by 
Modifying The Transmission in the Pupil Plane 



Figure 3. incoherent Optical Processor With Point Source of Light as an Input 
Object. For any Real Imaging System with A Circular Pupil, the 
Image is an Airy Diffraction Pattern 
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The image of a point source in an imaging system is called the impulse 
response or the intensity point spread function and is denoted here by |h(x)| 2 . 

It is directly analogous to the impulse response of an electrical filter?-® and 
completely characterizes the properties of the imaging system. The philosophy 
of incoherent processing is as follows. Since a circular aperture can modify 
the impulse response from a point to an Airy pattern, perhaps other pupil 
functions exist which could provide other useful impulse responses. If one can 
tailor the impulse response of the filtering system, one can hope to implement 
various kinds of filters. 

Shaping the pupil function to modify the impulse response has been studied 
for years by astronomers who wish to separate two nearby stars of widely 
differing magnitudes. If the secondary maxima could be removed, then the 
weak star would be distinguishable. In astronomy this technique is known as 
apodization 1 ^) 14 (removing the feet). The analogous radar problem is called 
side-lobe reduction. 

We are concerned, of course, with objects more complicated than simple 
points. Let us see how the image of such an object can be built up from the 
impulse response, 1,10 In a well-corrected (isoplanatic) system, 1 ® the only 
result of a lateral shift of the object point is a corresponding lateral shift of 
the image pattern. Thus, the response to 5(x 0 ~x) is |h(x-x)| 2 . Under this 
condition, the image of an extended object is the sum of impulse responses 
due to each point of the object: 

|i(x)| 2 = /dx 0 |o(x 0 )| 2 |h(x-x o )| 2 (8) 

This result is formally similar to the response of the coherent system with 
intensities replacing amplitudes. Thus, even in the absence of an explicit 
Fourier plane, linear filtering can be accomplished. 

Although the Fourier plane does not exist, we may nevertheless take the 
transforms formally. If we define 

IKx)| 2 <=>,^(to) 

|o(x)| 2 <=>^(co) (9) 

|h(x)| 2 <=> F(co) 

then Eq. (8) becomes 

,?(co) = (?(oj).C(co) (10) 


26 



where £(co) is the Frequency Transfer Function for the intensity spectrum. It 
prescribes the transmission of spatial frequencies of intensity through the sys- 
tem. In the special case where £(cj) = #'*(£ 0 ) the filter is matched to the object 
and the autocorrelation output is obtained, 


= / dcoO(co)0*(co)e _itJX 

(11) 

= /dx o |0(x+x o )| 2 |0(x o )| 2 

(12) 

= |o(x)| 2 © |o(x)| 2 

(13) 


In spite of the formal similarity of Eq. (8) to Eq. (2), the frequency trans- 
fer function £(co) is very different from H(co) . Figure 4 compares these two 
functions^ for the simple case of a circular pupil. Note that, whereas H(co) 
is constant out to the cutoff frequency, £(to), for the same pupil, is maximum 
at co = 0 and extends out to a cutoff frequency which is twice that for the co- 
herent case. This central peaking of jC(co) is an essential feature of incoherent 
processors , as we shall now demonstrate. 

£(co) is the Fourier transform of the intensity impulse response |h(x)| 2 
which response is non-negative. It may be expressed as the transform of a 
product. Thus, £(co) is the convolution of the transforms of h(x) abd h*(x) 

Since H(co) is the transform of h(x) and H*(-co) is the transform of h^x), it 
is easily shown that £(oj) may be expressed as 


jC(cj) =./ dxh(x)h*(x)e"" iuJX 

(14) 

=J dw 0 H(co+w 0 )H*(a> 0 ) 

(15) 

= H(co) ® H*(co) 

(16) 


In other words, £(w) is restricted to be a member of the class of autocorrela- 
tion functions, and this restriction arises solely from the requirement that 
the impulse response |h(x)| 2 be everywhere non-negative. Since the maximum 
value of an autocorrelation function must occur at co = O, the incoherent proces- 
sing system must have its maximum frequency transmission at u> = 0 , High- 
pass filtering is thereby precluded. Contrast enhancement is not possible. 
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Direct Pupil-Plane Masking 


In spite of this limitation, many, filtering operations can be implemented if 
the proper pupil function can be realized to produce the |h(x)| 2 desired. The 
simplest way of implementing a pupil function is by means of phase and/or 
amplitude transparencies located in the pupil function. 17)21 Sayanagi 17, has 
covered the pupil plane with randomly distributed discs of half-wave thickness. 
Figure 5 shows the very narrow frequency transfer function obtained by this 




H (to) 







77*' 


Coherent system 


U) 



Figure 4. Comparison of Spatial Frequency Transfer Functions for Processors with 
Circular Apertures. Left, Coherent System; Right, Incoherent System. 




Figure 5. Left. Incoherent Imaging System with Pupil Plane Covered with 

Randomly Positioned Discs of Half-wave Thickness. Right. Modified 
Frequency Transfer Function Due to this Modified Pupil Function. 
Dotted Line Shows Transfer Function for Unmodified Circular Pupil. 
Note: Replaces £ and v ' Replaces to . After Sayanagi . 
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Figure 6. Left. Image of Screened Object in Unmodi- 
fied Incoherent Imaging S/stem. Right. 

Image of Screened Object in Modified Sys- 
tem of Fig. 5. After Sayanagi . 

approach. When the halftone object (left) of Fig. 6. was located in the input 
plane, the processed image (right) of Fig. 6 was obtained because the spatial 
frequency of the dot pattern fell beyond the cutoff frequency of the modified 
imaging system. 

It is theoretically possible to prescribe a function |h(x)| 2 and proceed from 
there to implement a pupil function which will produce it. Born and Wolf 1- ’ 
show that the pupil function required to implement an amplitude function h(x) 
is the Fourier transform of the function h(x). To implement an intensity 
impulse response |h(x)| 2 the pupil-plane transmittance must be the transform 
of |h(x)|e l0(x) , where <p(\) is any phase function. One is still left with the non- 
trivial problem of choosing a 0(x) w’hich leads to a conveniently implementable 
pupil function. 

A much more convenient way to implement a complex transmission func- 

p O 9Q 

tion in the pupil function is the holographic approach. ’ 

Holographic Filters 

The system of Fig. 7 is due to Lowcnthal and Werts in France. A thin 
hologram is made as shown in Fig. 7b. A point source and a beam conveying 
information on h(x) impinge on a photographic plate. After development, it is 
clear that reillumination of the plate by the point source will reconstruct a 
virtual image of h(x) in its original location. If a lens is inserted at the plate 
such that h(x) is 24 in front, a real image of h(x) will appear 24 behind the 
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lens. Thus, an imaging system has been created between the input and output 
planes and this imaging system has the desired impulse response |h(x)| 2 . When 
an incoherently illuminated object (of essentially the same optical wavelength) 
is placed in the input plane as shown in Fig. 7a, the desired filtering will be 
accomplished in the output plane. An example of such a system acting as a 
matched filter for the character N is given in Fig. 8. The bright spots in the 
lower part of Fig. 8 are the autocorrelation functions which indicate recognition. 


Pupil plane 



— Hologram 




Aperture 

D- Photodetector 


2f 

(a) Incoherent holographic processor 


Pupil plane j 



Figure 7. Incoherent Holographic Optical Filtering System. Lower Figure Shows 
the Technique for Recording the Holographic Filter. Upper Figure 
Shows the Holographic Filter as Installed in the Processing System. 
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As in any imaging system, their locations arc dictated by the locations of the 
objects (here the N's) in the input plane. In the recognition phase, narrow band 
illumination at <>43 sa was obtained from a cadmium lamp. 

High efficiency in holograms requires a large angle between the two record- 
ing beams. Simultaneous implementation of a large number of filter functions 
(parallel optical systems) also requires filters to be located considerably off 
axis. To incorporate these features in t u* above system requires an extremely 
well corrected lens in each processor. A modified system which avoids this 
limitation is shown in Fiu. The function h(\)is here recorded with a 

converging rather than diverging reference beam. When the developed 
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LE FILTRAGE 

DES FREQUENCES SPATIALES 

EST UNE 

METHODE OPTIQUE DE 

^ TRAITEMENT DE L’INFORMATION 
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Figure 8. Input (Above) and Output (Below) from an 
Incoherent Holographic Optical Processor 
Designed to Recognize the Character N. 
Bright Dots Indicate Location of the Recog- 
nized Character. After Lowenthal and 
Werts 
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Hologram 




Figure 9. Lensless Incoherent Holographic Optical Processor. Both the 
Imaging Lens and tne Filter Function are Recorded on the 
Holographic Plate. Lower Figure Shows Method of Construc- 
tion of the Hologram. 
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hologram is reilluminated by a point source located as shown in Fig. 10b, the 
impulse response |h(x)| 2 is reconstructed to the right of the hologram in its 
original position. Thus, the holographic plate is now both lens and filter and 
the imaging system has the desired properties. No real lens is required in the 
processing system at all. Moreover, the lens used in the recording step need 
be corrected only for spherical aberration since it is used on axis. 

If a number of impulse responses are recorded on the same photographic 
plate, the resulting processor constitutes a bank of parallel optical filtering 
channels. Operation is in effect as if one replaced the lens of Fig. 7 by a fly’s 
eye lens, each lenslet of which had its pupil function modified to provide a 
desired filtering. As an example, we shall now consider a system designed to 
recognize alphabetic characters printed by a typewriter on ordinary paper. ^ 
We shall point out in passing a way of solving the problem of distinguishability 
that is common to all optical processors when they are used in patteim recogni- 
tion applications. 



Figure 10. Lensless, Incoherent, Holographic Character Recognition System. 

Filters are Matched to Features of Characters. Presence or Absence 
of Features is Determined by Optical Fibers and Detectors which 
Sense the Central Values of the Correlation Functions and by Thresh- 
olding Deyices. Simple Logic Circuits Infer Identity of Characters 
from the Feature Information Obtained. 
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Figure 10 shows the character recognition system. The input document 
containing character |o(x)| 2 printed or typed on ordinary paper is lighted by a 
laser beam (a convenient source of nearly monochromatic light). The reflected 
light falls on the holographic multifilter made as described above. The output 
correlations are located in the plane of the detection apertures where the peaks 
of these correlation functions are sensed by an array of optical fibers and detec- 
tors. The system is designed to recognize the subalphabet of Fig. 11. 

In the usual character recognition system, the filters are made to be 
matched to each of the characters of the alphabet. As a result, it is very 
difficult to distinguish very similar characters such as O and Q. The optical 
filters of the system of Fig. 10 are matched, however, not to the characters of 
Fig. 11, but to features of these characters. These features are chosen to be 
independent so that the recognition system need only decide the presence or 
absence of each and not the relative amplitudes of two very similar correlation 
peaks. Simple combinatorial logic then decides, on the basis of the features 
present, the identity of the unknown character. Although feature extraction is 
not strictly a part of optical processing per se, it is an essential ingredient of 
any system th^tt is to work with an alphabet of similar patterns! The set of 
features employed is shown in Fig. 12a superimposed on the character E. 

Figure 12b is an actual photograph of the impulse response of the multifilter. 

The photographic film was located in the aperture plane while a point source of 
light was placed in the input window. The five bars (the center bar of the top 
row is not a real feature but a positioning index) are shown as they appear when 
a point source of light is located in the document plane. With this system, the 
10 characters are still distinguishable even at document contrast ratios down to 
1.7:1. Further discussion of this system will be found in ref. 27. 

Although the laser is a convenient source of approximately monochromatic 
light, it is by no means essential. CRT input is possible^ if a phosphor such 
as one of those shown in Fig. 13 is employed. The ten characters above were 
recognized when present as transparencies backlighted by a CRT raster. The 
phosphor employed has the emission spectrum shown in Fig. 13a. The configu- 
ration is shown more clearly in Fig. 14 where the laser illumination of Fig. 10 
has been replaced by the CRT face. Although limited by phoshor brilliance, 
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Figure 1 1 . Subalphabet of Characters Used in Demonstrating the 
Recognition System of Fig. 10. 
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RELATIVE INTENSITY 
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(b) 

Figure 12. Features Employed in System of Fig. 10. a) 
Five Features Shown Superimposed on the 
Character E. b) Actual Photograph of the 
Impulse Response of the Recognition System . 
The Five Features are Shown Together with 
One Positioning Mark. 



Figure 13. Emission Lines of Two Rare-Earth Red Phosphors Commonly Utilized in 
Color Television. A CRT was Constructed Using the Phosphor of 
Fig. 13a. 
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Figure 14. Lensless, Incoherent, Holographic Character Recognition System 
Using CRT Illumination. 


hysteresis, aging and persistence, the CRT does allow the possibility of proc- 
essing any signal present as an electric current. Moreover, contrast enhance- 
ment may be achievable electronically. Unless operated in a raster mode, a 
serious drawback does arise from the tendency of the CRT to draw patterns of 
constant total light power rather than constant intensity. 

CONCLUSION 

The incoherent optical processor is capable of performing many of the 
desirable filtering operations normally associated with the coherent processor. 
Although it cannot achieve high-pass filtering, it can nevertheless process 
inputs present as diffusing or incoherently illuminated patterns which the 
coherent processor cannot handle. Thus, the possibility of processing CRT 
display images or real-time objects illuminated by sunlight may be considered. 
When the filters are implemented holographically, the optical system can con- 
sist simply of an input plane, a hologram plate and an output plane with no 
lenses needed at all. These properties make such a system an interesting 
candidate for on-board processing of earth satellite data. 
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Images (usually, but not necessarily, photographs) may be considered to 
be the record of events or conditions, and are generally used in one of two 
ways: Inspection and analysis by eye, where the preciseness of the brightness 
reproduction is secondary; and/or numerical analysis and data extraction, 
where the preciseness of geometry and/or brightness may be of paramount 
importance. It is quite likely that a given picture, although taken for a particu- 
lar reason, may not be in the optimum condition to fulfill its objectives. In 
addition, if a given picture is used for other than its intended original purposes, 
a completely new approach to the presentation of its data may be indicated. 
Therefore, it may be expected that image processing will be of benefit in 
optimizing the image use. 


Available in a practical sense are four different techniques for image proc- 
essing: Incoherent optical processing has been utilized for years in the photo- 
graphic industry, as exemplified by the darkroom manipulation for contrast 
manipulation, density, slicing, dodging, photogrammetric rectification and 
the like. Photographic manipulation of black and white separations of color 
images allows wide variations in the color produced. Multi-film sandwiching 
operations produce such effects as unsharp masking, difference comparisons 
and superpositions. 

With the advent of the laser, coherent optical processing has become 
popular in the last few years. Its main impact has been to give access to the 
Fourier transform plane for such operations as spatial frequency filtering and 
correlation. In addition, the production and reproduction of holographic images 
fall in this category. 

Analog electronic techniques have been widely used to vary the condition of 
electronic signals which when reproduced as images result in image alteration. 


*This research presents the results of one phase of research carried out at the 
Jet Propulsion Laboratory, California Institute of Technology, under Contract 
No. NAS 7-100, sponsored by the National Aeronautics and Space Administra- 
tion. 
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The most popular exemplification of this technique is the set of video adjust- 
ments (contrast, brightness, hue) found on every television set. In addition, 
horizontal and vertical image sharpening is accomplished during the transmis- 
sion of television programs. 

The modern digital computer has made practical processing techniques 
for handling non-linear operations in both the geometrical and intensity domains, 
various types of noise cleanup processes which are non-uniform over the extent 
of the picture and has made possible numerical analyses of pictures. The 
entire gamut of processing operations is available: affine or non-affine geo- 
metrical transformation, application of camera calibration data, intensity 
shading and contrast manipulations, one or two dimensional spatial filtering, 
coherent or lion-coherent noise removal, numerical data extraction, multi- 
picture comparison, etc. 

Our primary concern here will be with the digital processing, using as 
illustrations pictures processed by the Image Processing Laboratory of the 
Jet Propulsion Laboratory. The techniques for image processing have been 
developed primarily for processing pictures as returned from the NASA space 
vehicles (for example, Ranger, Surveyor, and Mariner) to the Jet Propulsion 
Laboratory. However, these same techniques, and to a very large extent even 
the same specific programs, have been utilized to process pictures from elec- 
tron and light microscopes, industrial and medical radiographs, Apollo pictures 
of the earth and moon, and various aerial photographs. Some of these will be 
illustrated later. 

It is useful to consider the image processing techniques to be grouped into 
three general areas depending upon the use to which the processed image is to 
be put: 

• Enhancement - The modification of subjective features of a picture to 
alter the impact of that picture on the viewer. In general, the enhance- 
ment processing will not produce a picture with numerically accurate 
data. 

• Quantitative Restoration - The application of camera calibration or 
other numerical data to the picture content so that the resulting pic- 
ture has point-for-point numerical meaning. The primary effort 
here has been the application of camera calibration data to remove 
camera artificats, although the removal of other contaminants such as 
atmospheric effects belongs in this category. 
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• Information Extraction - The conversion of the picture data into the 

information or decision required by the analyst, perhaps with a reduc- 
tion in the quantity of information required. 

Images taken on photographic film will show appreciable film grain noise 
if analysis of very fine detail is required. This effect is particularly noticeable 
in the high-speed film used to photograph very faint images. An example of 
this type of noise is shown in Figure 1, in which the upper left hand framelet 
shows considerable contamination of the picture content by the noise. In this 
processing sequence, a series of eight exposures of the same object was made. 
These were digitized independently and registered in the computer, following 
which 2, 4, and all 8 of the images were averaged in the computer. The result 
of the noise cleanup from this averaging process is shown in the other three 
framelets of Figure 1. 

One of the early applications for digital processing was the analysis of the 
roughness of the lunar surface. For this analysis, altitude contours of the 
lunar surface were derived from the (monoscopic) pictures taken by the Ranger 
spacecraft by utilizing the known photometric function (reflected brightness vs. 
viewing angle) (1). One of the Ranger pictures, its photogrammetrically 
rectified equivalent, and the altitude contours are shown in Figure 2. 

Since this analysis used the reflected brightness to determine surface slope, 
it is important that the brightness be accurate. Since television camera systems 
are not sufficiently uniform in their intensity response over the face of the pic- 
ture, corrections must be applied. For this purpose a brightness response 
function is derived for every pixel in the picture from a sequence of flat field 
pictures taken at different brightnesses. The results of this process is illus- 
trated by one of the Surveyor pictures, Figure 3. A demonstration of the 
variation in camera response across the test field is shown in Figure 4, in 
which are shown the flat fields as seen by a Mariner '69 camera at two different 
brightnesses. 

The true ground coverage plot of the contours shown in Figure 2 was 
produced by geometrically rectifying the incoming picture. This, and other 
geometric manipulations to be described below, is done digitally by defining 
to the computer the new desired locations of known points. Locations of all 
the intermediate points are generated by interpolation and the output picture 
generated. 

A major use for digital filtering at JPL has been for the enhancement of the 
high spatial frequency content (fine details) of the pictures. An example of this 
process is given in Figure 5, which shows the appreciable sharpening of a 
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Figure 1 . A Single Image Upper Left' Shows Appreciable Film Noise - Original 
Nebula Diameter Was Approximately 4mm. The Result of Computer 
Averaging of 2, 4, and 8 Independent Original Images is Shown in the 
Upper Right, Lower Left, and Lower Right. 
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c. Altitude Contours 
Derived from b 


Original Ranger 
Picture 


b. Photogrammetrical ly 
Rectified Picture 


Figure 2. Derivation of Altitude Contours from Photometry 


a. Before Correction 


b. After Correction 


Figure 3. Correction of Intensity of a Surveyor Picture 
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Figu rc * 4. Intensity of the Mariner '69 Camera to Flat Fields of Different Bright- 
nesses. The Contouring has been Produced by Removing the Three Most 
Significant Bits and Stretching the Remainder to Cover the Entire Bright- 
ness Range. Thus Each Contour Sequence Represents a Range of 8 
Digital Numbers of the Original Data. 



Figure 5. Enhancement of High Spatial Frequencies on a Surveyor Picture 







picture due to this type of filtering. Our approach to this process is discussed 
by Nathan (2) and illustrated in Figure 6. Consider the image of a star. Due 
to the geometry involved, this image should be essentially a delta function, 
having zero dimension. In practice, however, this image will have a finite 
diameter, due to light which should have impinged at the delta function point 
arriving instead in the nearby area. The job of the high frequency enhancement 
filter is to gather this nearby light and re-insert it at the delta function point. 

A ring of negative values around the central pixel will remove the unwanted 
light and a very high value at the center will re-insert it there. This effect 
and the numerical values of a practical filter are given in Figure 6, together 
with the equivalent effect in the spatial frequency domain. 


The most recent JPL space missions, the Mars flybys in 1969 and 1971, 
required, in addition to the above types of processing, a more complete noise 
removal (3). Because of the complexity of the coherent noise pattern (Figure 7a) 
filtering was done using two-dimensional Fourier transform techniques. The 
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Figure 6. Action of a High Frequency Filter in the Real and Fourier Domains 
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Fourier transform process produces a "picture" displaying, as brightness the 
spatial frequency content of the picture in two dimensions. A coherent noise 
will display itself as a geometrically arrayed series of bright areas as shown 
in Figure Sa. The first step in the filtering is to locate each of these areas. 
The inverse transform of the array of areas is the noise pattern itself 
( Figure 7b>, which may be subtracted from the picture directly. An alternate 
method of filtering is to suppress the areas containing the noise spikes in the 
Fourier domain and then to re-transform the picture-sans-noise spikes back 
to the real domain. The result is shown in Figure 8b. 



a. Original Picture b. Isolated Noise 

Figure 7. Mariner '69 Picture with Coherent Noise 




Fourier Transform 
of Picture of 7a 


Picture After Removal 
of Noise 


Figure 8. Removal of Coherent Noise by Fourier Techniques 



The difference in processing required for different uses of a picture is 
shown in Figures 9 and 10. Figure 9 is one of the Mariner '69 pictures which 
has been correctly photometrically rectified (i.e. , photometric calibration 
has been applied). The large difference in brightness between the south polar 
cap and the surrounding areas fairly thoroughly obliterates detail in these 
areas, although the numbers in the digital form of the picture are correct and 
useful for photometric analyses. To render more of the detail visible, the low 
spatial frequency photometry was deliberately destroyed by using a filter which 
greatly reduces the low spatial frequency content. This reduces the large 
black-to-white excursions and superimposes all of the fine detail on a more or 
less medium gray background. This "maximum diseriminability" picture has 
proven quite useful in the analysis of surface features where photometry was 
of secondary consideration. 

COLOR CONSIDERATIONS 

Operations in the color or multispectral domain have proven useful for 
both enhancement for human viewing and for numerical data extraction. 

Because the eye is more sensitive to changes in hue than it is to changes in 
brightness, a conversion of brightness to hue will render subleties of detail 
more readily visible. 

A step-wise conversion of this type is shown in Figure 11. In Figure 11 
is a picture taken by the S-158 Lunar Multispectral Photographs Experiment 
on Apollo 12, together with seven slice pictures. These slice pictures were 



Figure 9. Mariner '69 Picture Proc- 
essed for Photometric 
Restoration 



Figure 10. Mariner '69 Picture 
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Figure 1 I . Apollo S — 1 58 Image. Original, and Seven Platform Slices of Equal 
Data Quantity 
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formed by dividing the original brightness range into seven sub-ranges such 
that each sub-range contains the same number of data points. Each slice 
picture is then formed by printing its data points in black on a white back- 
ground. Each slice range may be reproduced in a different color and the 
ranges superimposed to give a rendition in which color represents brightness. 

A similar type of platforming is currently available in hardware which operates 
in real time on pictures obtained by television camera pickup. 

Manipulations of multisipectral data can be used to isolate areas of various 
spectral properties. Various combinations of visible and infrared multispec- 
tral imagery have been used for a wide variety of investigations in the areas of 
geology, meteorology, oceanography, agriculture, and forestry. The more 
successful of these investigations are those using computer-based multivariate 
analysis, although some success has been achieved using visually interpreted 
multicolor photography (4). 

Multispectral work will be illustrated here with two examples from the 
S-158 experiment. In the first example, two spectral bands are used to 
determine the relative redness -blueness of various areas of the lunar surface 
(5). 

Lunar color and its variation across the surface has interested planetary 
astronomers for many years. This interest has heightened with the growing 
weight of evidence, obtained from apcurate earth -based photoelectric photom-, 
etry, which points toward a positive correlation between color and composi- 
tional differences (6). 

Of course, the variations in composition are of paramount interest in lunar 
exploration. Once we have obtained ground observational data at several sites 
of differing color, it should be possible to extrapolate compositional color infor- 
mation to large areas of the moon which will not be sampled in situ. 

For this purpose, pictures of the same area were taken on black-white 
film through two color filters (Figure 12). After registration in the computer, 
the data was intensity corrected to log exposure, after which the pictures were 
subtracted, one from the other. The conversion to log exposure and the subse- 
quent subtraction produces a picture whose numerical value at a given point is 
a measure of the red/blue ratio at that point. The resultant picture thus 
represents the red/blue ratio rather than the brightness for the area covered, 
for the digital processing has eliminated the brightness component. Figure 13 
shows the red, gray, and blue areas each brought to saturation to indicate the 
respective locations. Color versions of these pictures are given in references. 
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For the flight pictures from S-15N, the extraction and presentation of 
color data was carried one step further to the simultaneous presentation of two 
color ratios (red ^green and blue green) derived from three input pictures. To 
do this, the hue ratios were assigned as the two axes of a Cartesian coordinate 
plane in which a unique color is assigned to each point. During the processing, 
it each picture element (pixel) location the red green and blue green ratios 
a n determined and used as entry points into the color plane. The required 
color for that particular red green, blue green intersection is then generated 
In computer and inserted at the corresponding pixel location in the output 
picture. The actual output of tin* computer process is a set of four color sepa- 
ration negatives which are to lie combined in a color printing process to pro- 
duce the output color picture covering the originally photographed area. Color 
versions of these are given in reference 8. 


An example drawn from a traffic motion study is shown in Figure 14. The 
left-most two framolets were taken a short interval apart so that moving 



Figure 12. Mare Imbrium Area of the Moon 


P.i.uv is reproduced 
h,u * 1,1 report li\ a , 
reproduction method to 
1‘i tti r detail. 


nt the 

I l iferent 
pro\ nle 


52 



vehicles occur at different places. The third framelet results from the regis- 
tration and subtraction of one original framelet from the other. In this, the 
freeway structure and stationary vehicles cancel and all moving vehicles are 
represented by an image pair. Thus, knowing the timing between the pictures 
and the spacing of the image pair, the vehicle speeds may be calculated. The 
right framelet results from displacing one of the originals from the other by 
an amount corresponding to, sax, the speed limit. In this case, all vehicles 
going at that speed cancel and vehicles going above or below this limit show as 
differential pairs. 

An example of geometric stretching of pictures for registration with other 
images (in this case a map) is taken from coherent sidelooking radar. The left 
framelet of Figure la is an original radar image after reconstruction to the 
real domain, with a computer grid superimposed. By knowing the aircraft 







Figure 14. Traffic Motion 
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Figure 15. Radar Images 






altitude and the geometry of the radar, the geometrical distortion caused by 
the sideways sweep may be calculated. The corresponding correction function 
may then be used to geometrically stretch the picture to produce true ground 
coverage of the image as shown in the center framelet. The right framelet is 
the same image softened by computer filtering and with computer superposition 
of contours from a topographic map to show the correlation between the radar 
and the mapped coverage. 

The final example illustrates the use of geometric stretching of pictures 
to produce an orthographic photomap from a sequence of computer mosaicked 
pieutres (Figure 16). In this sequence, five images from the Mariner 7 mis- 
sion have each been geometrically distorted to display the picture in an ortho- 
graphic projection. Following this, the individual frames were mosaicked by 
computer to produce the finished photomap. Details of the process are given 
in Reference 7. 

CONCLUSION 

The imaging processing techniques developed for enhancement and calibra- 
tion of the JUL imaging experiments has proven useful, not only for the original 
application, but has successfully been applied to images from a wide variety of 
sources. As computer processing becomes more available and the processing 
costs decrease, and as the techniques are improved to produce more and more 
useful results, the use of computer processing for enhancement of imagery will 
increase. 

As the techniques of parallel image processing improve, it is expected 
that some of the processes illustrated here will be so implemented. It is hoped 
that this discussion will prove useful in suggesting a group of processes which 
have been utilized, and which could benefit by some form of parallel processing. 
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Figure 16 . Five Frames (7N 11, 7N 13, 7N 15, 7N 1 7, and 7N 19) from the Mariner 
7 Mission have been Processed for Maximum Discrimibi I ity , Computer- 
stretched for Proper Geometry, and Computer-mosaicked to Produce 
this Orthophotograph of the Martian South Polar Cap 
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The demand for large capacity fast access time memory 
has increased to an extent beyond the reach of conventional 
magnetic memory technology. New approaches involving 
optical technology have been introduced to meet this chal- 
lenge. Using optic beam for addressing, the information bit 
size can be as small as optical wavelength, resulting in a 
packing density of 10^-10® bits/cm2. The ability to focus a 
beam at a distance eliminates the problem of crash and wear 
between the read-write transducer and the medium. With the 
introduction of inertialess beam deflectors, and the use of- 
page oriented technique such as the holographic storage, the 
access time of such a large memory may be reduced to 
microseconds. 

The basis for these promises is founded in the research 
progress made in recent years in the areas of materials, 
optical phenomena and laser applications. Of the various 
optical memory techniques advanced to date, magneto-optic 
and electro-optic approaches appear to be most promising 
and therefore will be reviewed in particular. In the for- 
mer case, the thermomagnetic effects induced by localized 
heating on a suitable magnetic medium from a focused laser 
beam is utilized for writing and magneto-optic effect is 
utilized for reading. The electro-optic writing techniques 
include the laser damage, the ferroelectric and ferroelectric- 
photoconductive effects. The read-out signal is derived from 
the electric discharge or the electro-optic effect. Besides 
these, other optical techniques such as the photochromic , 
thermoplastic, and amorphous semiconductor will also be 
reviewed. 

To make a mass memory system from these technologies, 
there are a number of essential considerations and trade-offs 
that require careful examination. Considerations such as 
the laser heating requirements, the depth of field for the 
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focusing optical system, the practical packing density and 
the problems associated with the signal-to-noise ratio in 
the readout have to be studied together. Trade-offs between 
the packing density, access time, data rate, and other im- 
portant parameters are of prime concern to the memory sys- 
tem design. These topics will be discussed with particular 
reference to a system using MnBi films. 

Beyond the bit-by-bit type memory, holographic 
memory system offers unique advantages and warrants 
further discussion. It is believed that the advantages of- 
fered by optical memory technique can be extremely useful 
for the onboard bulk memory and data processing needs in 
future earth observation systems. 
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The concept for an ERTS, the first in a series of earth observation satel- 
lites, was published in 1967. Between that time and the approval of the ERTS 
project in January 1969, the subsequent studies in 1969-70, and the implemen- 
tation phase which started in July of 1970, the data handling requirements for 
the system were being formulated. There were many problems in trying to 
assess these requirements as is the case in any interdisciplinary program of 
that magnitude. In particular it was difficult to clearly identify the users, 
determine their requirements for product types and to estimate the data 
volume . 

In this paper some of the data handling considerations that led to the 
formulation of the data processing requirements for the ERTS system will be 
discussed. The significant parameters that influenced these requirements 
are: (1) payload characteristics, (2) spacecraft characteristics such as orbit, 
attitude and attitude rates, (3) payload coverage, and (4) product types and 
quantities. Items 3 and 4 have had the most significant influence on the genera- 
tion of the requirements. 

PAYLOADS 

The ERTS imaging sensors consist of a return beam vidicon (RBV) camera 
and a multispectral scanner (MSS). The scanning geometry of the sensors is 
shown in Figure 1. 

The MSS images the surface of the earth in four spectral bands through ^ 
the same optical system. The scanner covers spectral range of 0. 5-1. 1 •*' 

micrometers. It uses an oscillating mirror which scans normal to the space- 
craft velocity factor. The spacecraft motion provides along the track progres- 
sion of the scan lines. The MSS detector outputs are sampled, codqjjl to six 
bits, and formated into a continuous data stream of 15 megabits per^second. 

Since the MSS is a line-by-line continuous scanning device, the geometry 
of MSS images recorded on film is influenced by spacecraft attitude rates in 
all three axes (pitch, roll, yaw). 


61 







The RBV system operates by simultaneously shuttering at 25 second 
intervals three cameras, covering the .475-. 83 micrometer range. Images 
stored on photosensitive surfaces of the camera tubes are read out by an elec- 
tron beam. To provide a video signal output, each camera is read-out sequen- 
tially, requiring a video bandwidth of 3. 5 MHz during read out. . 

The RBV is not sensitive to spacecraft attitude rates because each 
185 x 185 km image is stored on the photosensitive vidicon surface in a few 
milliseconds. Non-linearities on the sweep circuitry of the camera read-out 
electronics and boresight errors in the alignment between the 3 cameras re- 
quire geometric corrections on the ground that are different from those applied 
to the MSS images. 

When operating over a ground receiving station, the MSS and RBV data 
are transmitted in reel time to the receiving site and recorded there on 
magnetic tape. For operations not in view of a receiving site, 2 wide-band 
video tape recorders are included as part of the payload. Each recorder 
records and reproduces either RBV or MSS data upon command and each has 
a record capacity of 30 minutes. 

SPACECRAFT COVERAGE 

The spacecraft operates in a circular sun-synchronous, near polar orbit 
at an altitude of 908 kilometers. It circles the earth every 103 minutes, 
completing 14 orbits per day, and viewing the entire earth every 18 days . The 
orbit has been selected in such a way that the ground trace repeats the earth's 
coverage at the same local time every 18 days to within ±18 kilometers. . 

A typical one day ground coverage trace is shown in Figure 2. 

Figure 3 illustrates the sensor operation time required to cover the land 
masses of the earth as a function of season. The payloads would be required 
to operate between 80 and 200 minutes every day in order to completely cover 
the earth’s land masses. Due to the fact that such coverage requirements do 
not exist at the present time, that the surface of the earth is on the average 
50$ cloud covered, that sidelap of over 50 $ exists at above 60° North or South 
latitude and that the life of the onboard tape recorder, is limited, the average 
daily payload operation of 78 minutes per day has been selected as the basis 
for determining the data processing workload requirements. This includes a 
full real time coverage of the continental U. S. (18 minutes), and 60 minutes 
of tape recorded coverage. This in turn translates into a daily acquisition of 
188 185 X 185 km. scenes in each of the seven spectral bands (3 RVB and 
4 MSS). 
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Figure 2. Orbit-to-Orbit Coverage Variation 
DATA PROCESSING WORKLOAD 

The 188 scenes per day, taken in 7 spectral bands generate 1,316 individ- 
ual image frames per day or 3. 3 million frames per year. If these data were 
to be handled in digital form, a storage medium with a capacity of 10-H bits 
per day or 3. 6 X lO-*- 3 bits per year would be required* With practical for- 
mating considerations this volume of data would require approximately 500, 000 
reels of standard computer magnetic tape to store the master copy. The 
actual number of tapes required would depend on how many copies of each 
image were desired. The obvious conclusion that one comes to is that the 


*The above figures were derived as follows: 

a) RBV: 4 X 10 3 lines/frame X 4 X 10 3 samples/line = 1. 6 X 10 7 
samples/frame X 6 bits/sample = 10® bits/frame X 3 frames/scene 
= 3 frames/scene = 3 X 10® bits/scene 

b) MSS: 2. 6 X 10 3 lines/frame (185 km swath) X 3.3 X 10® samples/line 
= 8. 6 X 10® samples /frame X 6 bits /sample = 5 X 10 7 bits /frame X 4 
frames/scene = 2X 10® bits/scene 

c) RBV and MSS: 3 X 10® + 2 X 10® = 5 X 10 8 bits /scene X 188 scenes /day 
= lOH bits /day X 356 days /year = 3. 5 X lO* 3 bits/year 
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storage and distribution of all ERTS-A image data cannot be done in the form 
of computer tapes for practical reasons. Because one deals with image data 
and because of the majority of investigators in the various disciplines have 
been in the past dealing with aerial photographs, photographic film would be a 
more practical medium for recording and distribution of the image data. It 
has been therefore planned to distribute all of the ERTS data in the form of 
photographs and only deliver a relatively small portion (10$) of the data in the 
form of digital tapes to investigators who are equipped to handle it or who are 
developing new techniques for utilization of digital image data. 

DATA HANDLING FUNCTIONS 

Prior to the description of the various data handling functions that are 
required for the conversion of the spacecraft-telemetered sensor voltages 
into user products, definition of terms is in order, since nomenclature has not 
been standardized in this area. Key data handling functions are defined below: 

a. Data acquisition - reception of the sensor and housekeeping telemetry 
signals, demodulation, tape recording and transmission of these 
signals, tracking and commanding of the satellite. 

b. Data processing - conversion of video signals to a number of standard- 
ized user products and distribution of these to the users. This func- 
tion includes the application of in-flight calibrations to all data, 
tagging of all data with identification and location information based 

on spacecraft orbit and attitude, application of geometric corrections 
based on spacecraft ephemeris and/or use of ground control points. 

c. Data analysis - All discipline-related operations, such as generation 
of ocean temperature maps, land use maps, enhancement of linear 
features, extraction of information, correlation of data from many 
sources, etc. 

For ERTS-A existing NASA tracking stations provide the data acquisition 
support with the installation of special demodulation and data recording equip- 
ment. Three stations located in Fairbanks, Alaska, Goldstone, California, 
and Greenbelt, Maryland, will provide real time coverage for continental 
United States. Onboard video tape recorders will provide data recording capa- 
bility for the remainder of the required world coverage. 

Image processing functions such as conversion of tape-recorded video 
signals to photographic film, application of radiometric calibrations to the 
data and removal of geometric distortions from the imagery can best be per- 
formed in a central data processing facility prior to the distribution of the data 
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to the users. All of the required data such as spacecraft orbital parameters, 
in-flight and pre -flight calibration parameters are available in this central 
location. For this reason the data processing functions for ERTS are to be 
performed in a central data processing facility located at GSFC. 

All data analysis functions for ERTS are planned to be performed by the 
investigators in their facilities. The reasons that led to this decision were as 
follows: the interdisciplinary nature of the program would require a large 
array of analysis algorithms for conversion of raw images to such products 
as thematic maps, edge-enhanced images and various other special products. 
In addition to the sheer quantity of different products that would be required, 
proven algorithms that would satisfy the requirements of investigators in the 
various disciplines do not exist at this time. Many of these will be developed 
by the investigators who are participating in the ERTS-A program. 


The above considerations led to the implementation of an ERTS data 
management system shown in Figure 4. Payload video, housekeeping and 



Figure 4. Image Data Flow 
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tracking data are acquired at the data acquisition stations. Data processing 
is performed in the central data processing facility at GSFC. A variety of 
standard image products are delivered to the user community for analysis in 
their facilities. 

IMAGE DATA PRODUCTS 

The central data processing facility converts recorded sensor signals to a 
variety of standard film and computer tape products. In addition to the video 
tapes, spacecraft housekeeping telemetry and orbit tapes are required. All of 
the video-taped signals are converted to 70 mm film. Radiometric and geo- 
metric corrections based on pre-flight, in-flight calibrations, spacecraft 
altitude and attitude derived from orbit and housekeeping tapes, are applied. 
The 70 mm film masters are copied, enlarged to 9 1/2" transparencies and 
paper prints. Sixteen mm microfilm images are also generated. A portion of 
the images are also presented in the form of color composites. Available bulk 
film products are listed in table 1. 

Selected data are also formated into computer readable tapes. The RBV 
computer tapes are uncorrected. The characteristics and quantities of bulk 
tape products are shown in table 2. 


Table 1 


A. 


B. 


Standard Bulk Film Products 

• Application of pre-flight and in-flight radiometric calibrations 

• Application of geometric corrections based on S/C ephemeris data 
All Acquired Data: 

• 70 mm positive and negative transparencies 

i 

• 9 1/2 in. positive transparencies and paper prints 

• 16 mm microfilm (part of Data Catalog) 

20$ of Acquired Data: 

• 9 1/2 in. (10 in. ) positive color transparencies and paper prints 
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A portion of the images undergo additional processing which significantly 
improves the geometric accuracy of the images. In this process, points in the 
images whose location on the ground is precisely known are automatically 
identified and measured. This information is then used to fit the whole image 
to its proper location on the surface of the earth, thus improving the location 
accuracy. Products generated in this manner are known as precision products. 
Their quantities and types are shown in table 3. It should be noted that these 
products are available in both film and digital tape form. 

The ERTS data handling system has been set up to deliver the bulk black 
and white film products to the user within two weeks of data acquisition, i.e. , 

Table 2 


Standard Bulk Tape Products 

• Application of pre-flight and in-flight radiometric calibrations to 
multi -spectral scanner data. 

• 5^ of multi-spectral scanner data converted to computer readable 
tapes 

• 1$ of RBV camera data converted to computer-readable tapes 


Table 3 


Standard Precision Products 


• Application of pre-flight and in-flight radiometric calibrations 

• Application of geometric corrections based on use of ground control 
points 

• 5$ of acquired data: 

-9 1/2 in. positive transparencies and paper prints 
-9 1/2 in. negative transparencies 

-9 1/2 (10 in.) positive color transparencies and paper prints 
-computer-readable tapes 
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before the start of the new 18 day data acquisition cycle. This delivery cycle 
is constrained by several factors. Definitive orbit tapes for example are 
available 5 days after data acquisition. Video tapes are air mailed from the 
Alaska and Califoi’nia stations to GSFC. No attempt has been made in this 
program to provide data to the investigators in near real time. Two factors 
led to this decision: (1) the ERTS system is a research rather than an opera- 
tional one, (2) the probability of the satellite being over an area where a 
natural disaster, for example, is taking place and where near real time data 
handling might be of value is only 1/18. 

FUTURE REQUIREMENTS 

It is difficult at this time to estimate the trend of the data handling require- 
ments for future earth observation missions. Much experience should be 
gained after the launch of the first 2 ERTS satellites. It is possible, however, 
to predict some of the trends at this time. The trend to increase spacial and 
spectral resolution of sensors will continue. It is likely that only one multi- 
spectral sensor, such as the MSS, will be carried on future missions, with a 
higher spectral resolution, 7-8 spectral bands versus 4 on ERTS-A, with only 
a modest increase in spacial resolution. The 3 RBV cameras will be probably 
replaced with one or two panchromatic cameras and a significantly increased 
(factor of 4) spacial resolution. Limitations in existing RF data links will 
keep the down link payload telemetry in the range of 30-50 megabits per 
second. 

On the ground, essentially all data requested by the users will be geo- 
metrically corrected to the point that registration between data points of the 
same area on the ground, taken on subsequent coverage cycles, will be in the 
order of 1 resolution element. The increased radiometric accuracy ( 8 bits 
versus 5-6 bits for ERTS) will require all-digital processing of image data 
for a substantially larger portion of acquired data than is the case for ERTS. 
This, in turn, will require development of a new high density digital data 
transfer medium between the central data processing facility and the investi- 
gators. 

As promising data analysis algorithms are developed by the ERTS investi- 
gators, and as these algorithms are accepted by the investigator community 
in general, it will be possible to apply these to the data in the central data 
processing facility, thus eliminating the need to have the work duplicated in 
several investigators’ laboratories. 

The ERTS data handling approach, outlined in this paper, can be 
considered only a start of an exciting, dynamic process of improving and 
streamlining the flow of earth observation data to the world user community. 
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PARALLEL IMAGE PROCESSING FOR EARTH 
RESOURCES DATA 

Dr. Richai'd LeGault, Infrared and Optics Labs, 
Institute of Science and Technology, University of Michigan 


ABSTRACT 

Multispectral sensing has become an accepted tool for 
remote sensing, A major reason for the acceptance of 
multispectral sensing is the ability to automatically classify 
terrain objects. This automatic classification of terrain 
materials requires rather trivial computation for each 
resolution element but there are many resolution elements 
to be classified. The operational application of multispectral 
techniques presents a formidable computational problem. 

This talk will review the current classification algo- 
rithms and preprocessing schemes with emphasis on the 
computational aspects. The Cora Blight Watch experience 
of 1971 will be used to illustrate these computational 
problems. A parallel data processing technique adapted 
to multispectral classification is postulated. A functional 
outline and processing times for a parallel data processing- 
system will be presented. 
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THE MULTISPECTRAL APPROACH FOR PROCESSING 
EARTH OBSERVATIONAL DATA 

Dr. David Landgrebe 

Department of Electrical Engineering, and LARS, Purdue University 


ABSTRACT 

An alternate to image processing for analyzing the vol- 
umes of data to be gathered by earth observational satellites 
is possible by using so-called multispectral sensing followed 
by multivariant analysis techniques. This approach rests 
chiefly upon the use of spectral variation in the data although 
spatial and temporal variations can be used to derive the 
desired information as needed. 

This approach and developments associated with it will 
be described and illustrated with results from aircraft scan- 
ner data and multispectral space photography. 
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SPACE VEHICLE-BORNE APPLICATION OF 
OPTICAL DATA PROCESSING 

Joseph D. Welch, General Electric Company 
Valley Forge Space Center, King of Prussia, Pennsylvania 

INTRODUCTION N7S- 

Recently concern has been expressed on the capability of an advanced oper- 
ational Earth resources system to effectively utilize the very large amounts of 
data which can be supplied by its sensing elements. This concern has related 
to required bandwidths of the satellite-to-Earth data link as well as to the 
effectiveness of the total system in providing information to the ultimate users 
rapidly and in a form most useful to them. One approach to achieving an effec- 
tive operational system design which is deserving of investigation is that of 
processing much of the sensed data on-board the vehicle. This can have poten- 
tial advantages relative to reducing required bandwidths as well as making more 
effective use of the sensing capability by permitting some of the system deci- 
sion making to be accomplished by the on-board data processor. 

In considering alternative techniques for on-board data processing the 
techniques of parallel processing of imagery data has merit which is worthy 
of evaluation. By "parallel data processing" of imagery data we include any 
technique by which every element of an image is processed simultaneously. 

Most data processing techniques which can be considered under this definition 
tend to be of an optical type. Specifically techniques of coherent and non- 
coherent optical data processing are applicable candidates. 

This paper considers a model for vehicle-borne parallel optical data proc- 
essing which might be used as a key part of an advanced Earth resources 
satellite mission. The subject matter of this paper is related to an investiga- 
tion of "Advanced Earth Resource Information System Modeling" which the 
General Electric Company is presently initiating under contract to NASA 
(NASA Contract No. NAS 5-23090). 

Before discussing the operation of the On-Board Data Processor, it is 
well to first note some, of the typical ultimate applications which such an 
advanced processor might be asked to solve. Some of these functions together 
with typical practical uses are listed in Table 1. 

The table is only a small sample of the types of operations which could be 
potentially possible with on-board data processing. An imaginative approach 
to the program can readily uncover many more potential applications. 

Preceding page blank 
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Table 1 


Typical Objectives and Examples of the On-Board Data Processing 


Objectives 

Typical Example 

Image data compression 

Instead of sending pictures of corn 
fields, transmit the number of acres 
of corn together with the percentage 
of healthy corn. 

To transform the primary data into 
a form more readily useful to the 
practical data user 

Instead of displaying cloud and ocean 
patterns, interpret the high probability 
of fish concentration areas for com- 
mercial fishermen. 

To make on-board use of some of 
the sensed data to modify the mis- 
sion program 

To interpret some pre-selected out- 
puts of certain key earth observation 
sensors to turn on other sensors, or 
to change spectral sensitivity, or to 
change field of view, or to change the 
mode of data processing, or to pro- 
gram a sensor or vehicle roll search, 
etc. 

Image enhancement 

Based on deterministic or self- 
adaptive functions; select certain 
images for enhancement on-board the 
vehicle by optical means. 

To perform real-time "generic" 
pattern recognition 

The detection of a spiral formation 
might provide an on-board detection 
of a tornado and might be used to trig- 
ger detection search and automatic 
alarms. 

To perform real-time specific 
recognition 

On-board automatic recognition of 
specific earth surface features might 
provide navigation as well as relative 
location of other observed phenomena. 

Real-time spatial frequency 
analysis of images 

Determination by spatial frequency 
analysis of sea state, cloud and 
weather patterns, patterns associated 
With certain geological formation, etc. 
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Table 1 (Cont'd) 


Typical Objectives and Examples of the On-Board Data Processing 


Objectives 

Typical Example 

Real-time spectral analysis 
Image change detection 

Agricultural crop conditions. Nature 
of air and water pollutants. 

Sampled observation or change detec- 
tion of major time varying events 
(e. g. , monitoring progress of forest 
fire or flood on successive orbits). 


Based on considerations of these applications, it is expected that the 
ultimate potential benefits of such a processor could include: 

1. Simpler and less expensive ground stations. 

2. Lower downlink bandwidth and, consequently, lower power required 
on board the vehicle. 

3. More rapid data availability in more convenient form. 

4. Better utilization of vehicle, including all qf its sensory capacity as a 
result of self-adaptation to the stimuli of the sensed data. 

As presently conceived, an ultimate on-board data processor of this type 
would be an integrated entity which performs many interrelated functions. It 
seems appropriate to discuss approaches which appear to have merit. 

For one promising approach, it is conceptually convenient to think of a 
typical on-board data processing system as consisting of two interrelated parts: 

1. A "High Data Rate Processing Section" (e.g. , Parallel Data 
Processor). 

2. A "Low Data Rate Processing Section". 

Briefly stated, the High Data Rate Processing Section will operate, in a 
parallel data processing mode, to reduce the very high data rate sensor outputs 
to more meaningful, lower data rate outputs. The high data rate processor 
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receives the data from the external sensors and processes it in accordance 
with a program as directed by the controller function of the Low Data Rate 
Processor. 

The Low Data Rate Section will accept these outputs plus stored and trans- 
mitted commands. It will serve as a logic module to control the operation of 
High Data Rate Processing Section and also to control other related vehicle 
subsystems and will select and encode the desired compressed data to be trans- 
mitted to earth. 

It is evident that these two parts of the on-board system are inter- 
connected by a variety of possible feedback loops. 

Conceptually, the vehicle-borne part of the system might be described by 
the very simplified control loop of Figure 1. The essence of the system con- 
sists of advanced parallel data processing components interconnected by 
appropriate control and data loops. 

In order to achieve the required high data l’ates, the on-board "High Data 
Rate Processing Section" will typically be an application of a parallel optical 
data processor with capabilities for data processing in both the spatial and 
spatial frequency domain. 



HIGH DATA HATE 
"PARALLEL 
PROCESSOR" SECTION 


LOW DATA RATE 
CONTROL AND 
OUTPUT DATA 
SECTION 


Figure 1 . A Simplified Concept of the On-Board Data Processing System 
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Based on this premise and on the assumed existence of a flexible Low 
Data Rate Processor, an overall typical system can be blocked out as shown 
in Figure 2. 

This system is presented merely as being typical of a general approach. 
Different data needs and/or components availabilities, for example, will indi- 
cate a variation in the processor. Some of the primary things which should 
be mentioned concerning Figure 2 are: 

• The entire system is characterized by a network of data and command 
loops interconnecting the various functional components of the system. 

• Some of the system functions are performed on the earth, but for this 
advanced system, most automated functions are performed in the 
spacecraft. 

• Those familiar with optical data processing will recognize the func- 
tions and components of the High Data Rate Section of the on-boai’d 
processor. 

• The Low Data Rate Section is clearly the on-board "nerve center" of 
the system. 

• The system, as shown in this figure, should require no moving parts 
on the vehicle nor any sequential scanning of continuously high bit 
rate data. 

• ! This system provides the user on the ground with the data in a form 

most usable to him with the minimum of operations required. 

• The system, as show, is presented only as an example. Many 
components and interconnecting command and data loops could be 
added, deleted, or modified based on judgments of user needs, tech- 
nical feasibility, costs, and varied opinionated engineering judgments. 

• With regard to the Data Storage Unit, many points of view can be 
taken. It could be conceived as a part of either the High Data Rate 
Section or the Low Data Rate Section, more likely it will have func- 

, Rons in both areas. 

• This figure is a nice picture— but how does it work? The answer, as 
far as the overall system is concerned, is largely application- 
oriented. 
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The most effective way of describing the concept shown is by way of some 
specific examples. Therefore, some cases of how a general system such as 
this one could operate are described: 

CASE (A) 

Problem 


Provide to a user a spatial frequency response of an image of a specified 
area in a specified spectral band (as might be useful, for example, in a geo- 
logical survey). 

Solution 

The details of the command, including its programming sequence, are 
assumed to be stored in the Data Storage Unit. The Command Unit provides 
electrical signals to components in accordance with stored or transmitted 
commands. 

In this case the Command Unit provides appropriate electrically encoded 
signals to the electrically alterable spectral filter and the electrically alterable 
spatial frequency filters. The spectral filter assumes the desired color re- 
sponse. The spatial frequency filter assumes a sequenced combination of 
multiplicable transparent annular rings and wedges in synchronism with the 
integrated optical-electrical readout signal. This "output" data obtained by 
conventional two-dimensional Fourier transformation techniques is "tagged" 
synchronously to provide the desired spatial frequency information, in the 
desired spectral band, for purposes of encoding and transmission to the user. 

CASE (B) 

Problem 

Recognize and locate a suspected large source of air or water pollution. 
Solution 

On the basis of a command transmitted from the earth station begin search 
operation when the space vehicle is near suspected "target". Proceed to search 
for the spectral response of the pollution by means of an electrically alterable 
narrow-band spectral filter combined with a self-adaptive threshold device. 
Monitor the optical electrical output signal for an unusually intense signal. 

This intense signal can be used to trigger a landmark correlation search by 
coherent o uical means using a stored file of electrically retrievable matched 
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spatial frequency filters. The correlation signal detected by the optical- 
electrical detector, together with on-board information on the location and 
pointing attitude of the space vehicle can provide information on the location of 
the pollution source. 

Furthermore, this cross-correlation interpreted in the parallel processor 
as a change-detector can confirm the addition of a major change (e.g. , the 
newly observed pollution signature). 

CASE (C) 

Problem 

Enhance selected images which were subjected to high spatial frequency 
contrast loss due, for example, to atmosphere effects. 

Solution 

Based on commanded signal, program an electrically alterable spatial 
frequency filter capable of "grey scale" transmissivity to assume the desired 
low spatial frequency suppression to compensate for high spatial frequency 
loss. Detect resulting signal with optical-electrical output sensor for encod- 
ing and transmission to Earth. 

An interesting self-learning improvement of this may be possible by pro- 
viding an idealized optically-stored MTF frame of reference on board the space 
vehicle as a self-adaptive guide to select the required optimum enhancement 
high pass spatial frequency filter function. 

It is hoped that these few examples will adequately describe the approach 
to the overall system. 

As noted before, data is provided directly to the user in a compressed 
form most useful to him. As indicated in this figure, it is up to the user to 
initiate new requests for data and to make beneficial use of the data. 

OTHER VARIATIONS OF THE ADVANCED DATA PROCESSING SYSTEM 

Many variations of the system not specifically incorporated in Figure 2 
are possible as may be dictated by specific problems. Some specific examples 
are: 

1. A Built-in Parallel Processing Reiteration Procedure . Such a system 
would be designed such that the "output" image plane could also serve 


82 


as the "input" image plane for a new iteration. This is shown in 
rudimentary form in Figure 3. This reiteration approach to parallel 
processing is analogous to its counterpart in serial processing. The 
reiteration logic and evaluation would be incorporated into the com- 
mand and sequencer. 

Applications of such a reiteration capability would include a wide 
range of self-enhancing, or self-learning operations where a given 
process is reiterated as long as improvement, according to some 
established and optically stored norm, is automatically observed 
relative to that norm. 

2. A Processor which has Capability for Optically Making Spatial 

Frequency Filters . This possibility has application, for example, 
when an unexpected phenomenon is observed and when it is wished 
to be monitored automatically on a periodic basis to detect changes. 



Figure 3. A Portion of an Advanced Data Processor 
Showing a Reiteration Loop Operation 
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3. Parallel Data Processing Applied to a Variety of Sensors . Although 
we have emphasized "parallel" type of sensor inputs <e. g. , image in- 
tensifies and optical-to-optical interface devices) it is evident that 
optical data processing can be applied to a wide range of other sensors. 
In the case of serial read-in types of sensors, a channelized mode of 
parallel data processing may be most appropriate. 

FUTURE SYSTEMS 

The possible development of an advanced system based on unexpected user 
need and technology potentialities has been discussed. As progress in an 
imaginative, and yet realistic approach is made, a great analogy between 
advanced two-dimensional parallel processors and conventional serial digital 
computers can be foreseen. Entire images of the parallel processor become 
the analog of binary words of a serial computer. Extending this analogy to the 
design of parallel processing systems could lead ultimately to computers of 
extraordinary capabilities. 

THE "OUTPUT" OF THE OPTICAL PROCESSOR 

Here we consider the electro-optical readout of what appears on a defined 
output plane of the optical processor. Exactly what appears is a function of the 
specific processing task being pursued. Typically, it may be: 

1. One or more correlation functions indicating the correlation response 
of two images. 

2. A spatial frequency pattern of an image. 

3. A sequence of multispectral images. 

4. A selected enhanced image. 

This data may be used for purposes such as: 

1. An input to further optical iterative computations. 

2. An input to an on-board self-programming or self-learning operation. 

3. Encoding for downlink transmission. 

This section is included to point out the importance of this readout operat- 
ing at relatively high speed and preferably in a parallel operating mode. 
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THE "LOW DATA RATE PROCESSING SECTION" 


Since the data rate output of the High Data Rate Processing Section will not 
always be high in this section, parallel data processing, although not neces- 
sarily completely ruled out, will not necessarily be a feature of this section. 
Indeed, at some point before transmitting the data to earth, the resultant low 
bit rate data will be converted to serial format. 

It is likely that much of this part of the processor and its inputs and out- 
puts will operate with low data in analog or digital form as may be appropriate 
to the specific function. Although the data rate of this section may be relatively 
low, for many missions a high capacity data storage may be required. This 
storage device might be logically considered part of either the High Data Rate 
or Low Data Rate Section of the On-Board Processor. 

Consider next some of the logical operations of the Low Data Rate Process- 
ing Section. Typical examples of functions of this data processing section are: 

1. Interpret ground commands and/or stored programs to select 
operating mode of data processing systems and operating parameters 
of each component. 

2. Accept parallel readout of High Data Rate Section and operate on it as 
may be appropriately commanded. 

3. Typical operation on above read-out may be: 

a. X-Y interpolation and point center detection of correlation peak. 

b. S/N evaluation of correlation function, 

c. Serializing data of certain selected spatial frequency spectral 

images (preparatory to encoding for downlink). 

d. Serializing of a limited number of selected enhanced images. 

e. Interpreting output data for command variations such as: 

r i I 

: ' , 

• | Adjustment of threshold device 

IT 

• | Adjustment of gain or other "search" parameters of critical 
' components of system 

• Turning on or off of sensors, changing fields of view, etc. 
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f. Providing automatic programming, including decision-making by 
progressive logic with respect to future operation of the system 
based on both output data and internal commands. 

4. Evaluate raw and processed data in relation to user needs and make 
decision to process data further and/or to transmit data in a selected 
format to Earth via encoder and downlink. 

5. Ultimately, to approach a self-learning functional operation such that 
a more complete automatic operation of the system is to be ultimately 
feasible. 

Although these low data rate date processing functions are expected to be 
initially performed by electrical signals and solid-state components, far more 
advanced techniques such as very compact thin film optical data transmission 
and computing techniques should be considered for even more advanced 
missions. 

CHARACTERISTICS OF KEY RELATED TECHNOLOGIES 
AND COMPONENTS 

An understanding of key components, their characteristics and problems 
as well as some projection of their status and quantitative performance a 
decade hence, is of key importance to any investigation of an "An Advanced 
Earth Resources System Model". 

Most existing devices do not have all of the desired performance features 
to the degree which is expected to be needed for a high capacity processor of 
the type needed a decade hence. However, much effort on many related tech- 
nologies is underway to continually improve components and technologies. 

In discussing these critical items we will try to list the desired features 
of the specific devices. Although we will not repeat this fact for each item, 
it is evident that desired potential features of all components include: low 
power requirements, lightweight, small volume, and long life reliability in a 
space flight environment. 

OPTICAL-TO-OPTICAL INTERFACE AND OTHER INPUT 
INTERFACE DEVICES 

In its most readily conceived function, this is a device which receives a 
non-coherent optical input image and which is usually readout with a local 
source of coherent illumination. 
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The term "and other Input Interface Devices" is added so as to include 
other types of input interface devices such as various types of passive or active 
radiometers which may come into greater favor in the next decade. Hopefully, 
even these devices will be developed having parallel data input capability. 

An ideal optical-to-optical interface device should have these character- 
istics; 

1. High sensitivity in a desired spectral band (ultimately an electrically 
selectable spectral response would be attractive). 

2. High resolution. 

3. Very rapid in-place development. 

4. High modulation efficiency. 

5. Very short or controllable relaxation time constants (i.e. , automatic 
or controllable erasure). 

6. Low noise characteristics. 

7. High contrast rati,o potential. 

8. Capacity for indefinite recycling. 

These various interface devices include both amplitude and phase media. 

It should be noted that strides are taking place in the area of "parallel" 
operating image intensifiers. In some cases these can permit the practicality 
of an interface device which would otherwise have inadequate sensitivity. One 
can also foresee some realistic prospects for integrating the function of an 
optical-to-optical interface device and an image intensifier in a single device. 

As discussed, parallel operating image intensifiers can have significant 
merit in permitting the use of certain optical-to-optical interface devices 
which may appear attractive in all respects other than sensitivity. For this 
reason the General Electric Company has pursued an investigation of various 
types of image intensifiers. 

Some of the newer types of "distortionless" intensifiers appear very 
attractive for optical data processing. 
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In addition to a prime function of providing significant optical gain, certain 
types of image intensifiers can also provide these useful functions: 

1. High speed electronic triggering (desirable for certain types of optical- 
to-optical interface devices which tend tq be cyclic in their operation). 

2. "Zoom" (or variable magnification). This feature can permit a high 
speed magnification change or magnification search which may be 
desirable for certain types of data processing operations. 

ELECTRICALLY ALTERABLE SPATIAL FREQUENCY FILTERS 

Reference is made here to both amplitude and phase filters as well as to 
complex filters. The object of these types of devices is to provide a rapid, 
no-moving parts means for changing spatial filters. Spatial filters of interest 
include simple ring and wedge amplitude filters as well as filters having much 
greater capability (i.e. , electrically alterable complex filters). 

In principle, an electrically alterable complex filter can be accomplished 
in various ways, most of which fall in one of these categories: 

1. Electrically alterable phase medium arranged multiplicably with an 
electrically alterable amplitude medium. 

2. Electrically alterable, carrier modulated amplitude medium. 

3. Electrically alterable, carrier modulated phase medium. 

These last two carrier types might be described as holographic in their 
approach and some question may arise as to their optimum utilization of the 
available space bandwidth product of a given medium. 

ELECTRICALLY ALTERABLE SPECTRAL FILTERS 

Most multispectral imagery to date has utilized multiple filters (as in a 
filter wheel), multiple sensors, or other similar devices which involve moving 
parts or redundant equipment. There may be considerable merit in obtaining 
multispectral spatial frequency images as well as more conventional multi- 
spectral images . 

This seems to be an area for considerable investigation. The possibility 
of an electrically alterable spectral filter has, for example, great appeal as 
an element in a parallel optical data processor. If the center spectral frequency 


88 



could be tuned and also the spectral bandwidth could be adjusted electrically 
then this would indeed be a very powerful tool. 

SPATIAL FREQUENCY FILTER SELECTION AND 
RETRIEVAL TECHNIQUE 

This represents an alternative, or perhaps a complement to the 
electrically-alterable filter approach. In this present approach the spatial 
filter is stored optically and retrieved optically. 

At least two approaches come to mind: a coded multiplexing approach 
and a retrieval by optical modulator. 

A good example of filter multiplexing is described by Gorstein, Hallock 
et. al.* They have, for example, described the encoded superimposed multi- 
plexing of at least 20 independent star field spatial frequency images. This 
is not by any means believed to represent a limit of the approach as imposed, 
for example, by the space bandwidth product of the medium. Decoding the 
multiplex filter can be accomplished, for example, by reading out the location 
of a second correlation image on the output plane. Several variations of this 
intriguing technique are evidently possible. 

The second approach would make use of high density image type of data 
storage (e.g. , holographic storage) coupled with nondestructive laser beam 
readout techniques, for example. 

A certain number of high density data storage and retrieval devices are 
known to be under development including at least one under NASA contract. 

OPTICAL-ELECTRICAL READOUT DEVICE 

The function of this device is to convert an optical electrical signal appear- 
ing in some defined output plane of the parallel optical processor to an electrical 
signal. Consistent with the requirement for high speed parallel operation this 
type of device should be one which involves no moving parts or scanning of any 
type. 

The nature of the readout device will depend on the nature of the computa- 
tional operation being performed. For example, the readout of a two dimen- 
sional spatial frequency analysis could be conveniently accomplished in the 

^Gorstein, M. , Hallock, J.N. , and Valge, J. , "Two Approaches to the Star 
Mapping Problem for Space Vehicle Attitude Determination, " Applied Optics , 
vol. 9, No. 2, Feb. 1970. 
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spatial frequency plane by two assemblies of solid state detector elements 
shaped in the form of rings and wedges to respectively provide magnitude and 
direction of the spatial frequency content. 

Readout of more complicated images such as selected locally enhanced 
spatial images might ultimately be accomplished by a two dimensional solid 
state detector array. 

IMAGE ROTATION DEVICE 

In certain types of image data processing it may be desirable to provide 
a rotation of an input image about the optical axis of the optical data processor. 
One such example is where a rotation orientation or a rotation search is needed 
to satisfy a function of a matched spatial frequency search and recognition 
operation. 

Another example might be a particular approach to spatial frequency 
analysis where it might become attractive to rotate the image about the optical 
axis rather than use wedge spatial filters and/or wedge electro-optical readout 
sensors. 

Several means for accomplishing these rotations are possible including 
mechanically rotating the input image, or the spatial filter or rotating an image 
rotation prism such as a Dove prism or other similar prisms. 

It would be more attractive, however, to devise a parallel electrically 
alterable image rotation device which requires no moving parts. Some tech- 
nical approaches for accomplishing this have been conceived. It is hoped that 
practical techniques for accomplishing this in a compact optical data processor 
will become available. 

ULTRA HIGH DENSITY DATA STORAGE DEVICES 

Several such devices, both two dimensional and thrqe dimensional, are 
known to be under development. The optical approaches appear to be domi- 
nant. 

The major problems presently appear to be in read-in and read-out— 
particularly nondestructive read-out. 

The continued successful development of these devices will be advantageous 
to the pursuit of advanced parallel data processing. One specific example 
would be the high speed storage and retrieval of spatial frequency filters. 
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The continued investigation of high density data storage devices is of key 
importance. 

SUMMARY 

On-board parallel optical data processing is a promising approach to the 
data reduction problem which will be associated with very advanced types of 
Earth resources missions. 
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A GENERAL APPROACH TO SOME SPIN SCAN 
GEOMETRY PROBLEMS 

W. Swindell 

University of Arizona, Tucson, Arizona 
ABSTRACT 

The Pioneer F and G missions carry a spin scan imag- 
ing telescope for the purpose of obtaining two color images of 
Jupiter and its satellites. Although these are specific mis- 
sions, the problems are general in nature and have required 
a general approach in their solution. First it has been nec- 
essary to create special software that can rectify the other- 
wise distorted image that would result from the imaging 
geometry. The software is completely flexible in that the 
shape, size, tilt and rotation rate of the viewed (ellipsoidal) 
object are arbitrarily variable as are the clock and cone 
angles, stepping rate, spin axis orientation, spin rate, sampl- 
ing rate and trajectory of the imaging telescope. This pro- 
gram thus provides a useful tool for investigating the suit- 
ability of spin scan techniques for any flyby or orbital 
situation. Examples of scan maps will be shown. Second, 
the presence of curved scan lines (unlike TV systems) pre- 
sents special problems in the image data processing with 
respect to enhancement and alleviation from the effects of 
noise. Third the curved scan lines create a special problem 
when it comes to filling a display matrix. The nature and 
method of solution of these problems will be discussed. 

This work was supported by NASA /AMES Research Center, 
Contract NAS 2-6265. 
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THE GODDARD SPACE FLIGHT CENTER PROGRAM TO 
DEVELOP PARALLEL IMAGE PROCESSING SYSTEMS 

David H. Schaefer 

Head, Computer Technology Section, Goddard Space Flight Center 


My group has a program to develop hardware to accept a "real" image in- 
put, handle it as an entity in real time, and spit information— not an image, 
mind you— but spit information out the rear end of the system. 

The title of this conference contains the phrase "parallel image proces- 
sing". By our definition, parallel image processing is image processing where 
all points of an image are operated upon simultaneously. Our interest in paral- 
lel systems stems from the belief that, in the long run, only parallel image 
systems will be able to meet the very stiff speed and reliability requirements 
that will be demanded for image data processors aboard advanced spacecraft. 
Some participants at this conference have stated in discussions that the word 
"parallel" is simply another word for "optical. " 

I don’t believe that this is really true. With all the promises of large 
scale integration one would think that LSI arrays, if they were living up to 
their expectations of 10 years ago, would today be able to perform many compu- 
tations in parallel. We should be able to have a matrix of photo diodes directly 
connected to a matrix of logic and be able to perform "parallel image proces- 
sing. " Under our program the development of such large scale integration 
techniques is being encouraged. 

There is another method of parallel image processing that is not optical. 
Here one converts an image into a modulated beam of electrons , and manipu- 
lates this with the tricks that are possible with electron optics. We are sup- 
porting the development of techniques of this sort. 

Parallel image processing isn't necessarily optical processing, but, of 
course, optical methods are included. We want to develop both automatic 
non-coherent and automatic coherent optical image processors. Jim Strong, 
in another paper, is going to show experiments conducted here at Goddard 
where logical operations are performed on images. These experiments 
utilize non-coherent cross-correlation techniques. Dr. Husain- Abidi’s work 
with coherent optical systems will be reported in his paper. 

Under the Goddard program we want to bring all parallel image proces- 
sing techniques together: electronic methods, non-coherent optical methods 
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and coherent optical methods. We envision hybred processors using a combina- 
tion of techniques. Our program is dedicated to developing a variety of auto- 
matic parallel image processing techniques so that by the proper use of all 
techniques, image processing can be carried out on board spacecraft. 

Our program has two facets to it: 

The first facet is to develop the technologies that must be developed if we 
are ever going to perform the image processing tasks we talk about. Let me 
show you several examples. All these examples, it so happens, have their 
application in automatic coherent systems. 

The first example of technology development is our funding of devices to 
convert a non-coherent input image into coherent form. Hughes Research Labs 
have a contract to develop a liquid crystal device that will do this job. Before 
this talk is over a motion picture of this converter in operation will be shown. 

In the following paper the device is described in detail. 

We have a contract with ITEK to build a similar device. They utilize 
Bismuth Silicon Oxide, a material with both birefringent and photo-conductive 
properties. The converter has just very recently been delivered. 

Electrically alterable filters for use in the Fourier transform plane have 
been developed by ITEK under our funding. The filters are in the form of 
wedges and rings. Figure 1 shows the wedge filter. The filters are liquid 
crystal devices, and the white areas in the figure are areas that are in the 
scattering mode. In the figure there are three transparent wedge pairs. In 
preliminary experiments these filters have proven to be most useful and we 
are obtaining interesting results with them. 

Westinghouse has developed sensors that can electrically sense in the 
Fourier transform plane. These are in the form of photo-conducting rings 
and wedges. 

Along with developing technology, we are also developing the theory and 
methods of using these new systems. 

For instance, Yale University, under Dr. Richard Barkers direction, is 
undertaking both materials work and work in optical techniques, including 
techniques of performing spectral analysis in a parallel manner. 

Dr. Sing Lee, of Carnegie Mellon, has a grant from us to investigate 
advanced methods of coherent processing. 
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A rather small contractual arrangement with Avco Corporation has been 
recently concluded. We asked them to come up with a Fourier transform filter 
hologram that would recognize any kind of a spiral. If the image input is a 
spiral, you get an output. If the image input is something other than a spiral, 
you aren't supposed to get any output. This was a small effort and the results 
are not spectacular, but they are interesting. This work, we feel, will lead to 
a way of recognizing objects described in generic terms. 

Joe Welch yesterday indicated that General Electric has a contract to 
model advanceu Earth Resource information systems. This contract is just 
starting. Hopefully, it will indicate what kind of devices we are going to ulti- 
mately need for certain Earth Observation tasks. 


We are giving assistance to Fred Billingsley of JPL. He is going to take 
some of his serial algorithms and convert them into parallel algorithms so that 
certain of his operations can be performed in a parallel manner. 



Liquid Crystal Electrically Alterable Wedge Filter 
for Use in Fourier Transform Plane 
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This morning I was wondering how to give you a progress report of how 
far along we are in developing real-time parallel image processing systems. 

It seemed like the best thing was to show you a system that we have built up in 
our lab using some of the devices that we have gotten from the people who have 
been doing hard work for us. We are going to talk about a very primitive co- 
herent optical system. 

Our aim, as I said, is to have, as input, an image, and to obtain as output, 
information. 


For the system we are going to demonstrate, our "image in" is simply a 
black ruler (figure 2). The "information out" is a whistle indicating that the 
ruler is horizontal. 



Figure 2. "Input" to Experimental System 
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The first step is to focus the image on an image intensifier tube. In this 
particular experiment we were anxious to show that an image intensifier could 
be used to activate the non-coherent-to-coherent converter. 

The image of the stick on the output of the image intensifier tube goes 
through a beam splitter and is then focused on the interesting looking gadget 
at the right in figure 3. This piece of hardware is what we call our "OTTO' 
device for "optical -to-optical"— with a slight stutter. As you realize, this is 
the device that is going to change the non-coherent image into a coherent 
image. The laser is out of the picture at the right. The laser beam travels 
from the right, through the OTTO device and beam splitter, and on to the lens 
system at the left of the figure. 


Dr. Jacobson in the next paper is going to fully describe the OTTO device. 
Let me give you a very general description. It is a liquid -crystal cadium- 
sulfide sandwich. In the areas upon which non-coherent light falls the cadium 



Figure 3. Experimental Setup. The OTTO Device At the Right 

Hand Side of Figure 

Ihis pa Re is reproduced at the 
I'a. k of the report In a different 
reproduction method to provide 
hotter detail. 


i) ( J 




sulfide becomes a conductor malting the liquid crystal behind that particular 
area a scatterer. Other areas remain good light transmitters. Coherent light 
is, therefore, scattered out of the system in areas of non-coherent illumination. 
The output is, because of this, the negative of the input image. The non-coherent 
black stick on a white background is converted to a coherent red (from the He-Ne 
laser) stick on a black background. 

After conversion the Fourier transform of the scene is formed and 
focused on the photoconductor wedge sensor shown in figure 4. Whenever the 
stick is horizontal, no matter how it is translated up, down, or sideways, the 
Fourier transform will be a vertical line. By monitoring the two vertical 
wedges we have an electrical indication of when the stick is horizontal. The 
output from the two vertical wedges activates the whistle. 

We have here a parallel image processing system where the input is a 
non-coherent image, processing is pei'formed in the coherent optical realm, 
and the output is an electrical signal. Our prime reason for putting this 
together was to exercise the OTTO device in a very simple system. 

The sound motion picture that follows will show the system in operation. 
Certain limitations of our presently available devices, especially the limita- 
tions due to the slow time response of the OTTO devices on hand, will be 
evident to you. The movie will, however, show our first attempt at building a 
real-time parallel processor using cohei’ent techniques. It is our hope and 
belief that from such modest beginnings we will be able to develop hardware 
that will help the country in its many image processing tasks of the future, be 
these tasks on the ground or aboard spacecraft. 
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TEE LIQUID CRYSTAL LIGHT VALVE, AN OPTICAL-TO- 
OPTICAL INTERFACE DEVICE 

A. D. Jacobson, T. D. Beard, W. P. Bleha 
J. D. Margerum, and S. Y. Wong 
Hughes Research Laboratories, Malibu, California 90265 

INTRODUCTION 

The subject of this discussion is a new device which is called, generically, 
a photoactivated liquid crystal light valve. ^ But it is more affectionately known 
as OTTO, which is an acronym for optical-to-optical interface device. 

FIGURE 1— STYLIZED SCHEMATIC OF OTTO IN USE 

Very briefly, as shown in Fig. 1, the device is designed to transfer an 
optical image from a noncoherent light beam to a spatially coherent beam of 
light— i. e. , a laser— and to do it in real time. The purpose for doing this sort 
of transfer operation is to input the optical image from the real (i. e. , sunlit) 
world or the world of CRT's into the highly refined laser illuminated world of 
a coherent optical data processor. In the past this task has been carried out 
primarily using silver halide film. 2 From most standpoints, silver halide 
film is ideal for this purpose. It has high resolution, excellent contrast, 
adequate dynamic range, sensitivity and durability. Its main drawback, how- 
ever, is that it requires wet, chemical processing. From an operational 
standpoint, this requirement is an overwhelming limitation. Primarily be- 
cause it requires intolerable time delays between exposure and use— from tens 
of seconds up to substantial fractions of an hour depending on the type of film 
and the type of processing. In order to build practical systems for most 
applications it is crucial that the data input operation be performed much 
faster than can be done with conventional film. In fact, we feel that the absence 
of a real or quasi real-time input device has been one of the major factors 
impeding the development and use of coherent optical data processing systems. 
Unfortunately, it has proven to be very difficult to find a practical device (with 
emphasis on the word "practical") that offers both real-time performance as 
well as the full spectrum of optical qualities required by coherent optical data 
processing systems. In fact, to our knowledge, there still is no such device 
available on an off-the-shelf basis. The device that we describe below is a 
new candidate for that role. 
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LIGHT VALVES 

DEVICES WHOSE LIGHT TRANSMISSION CHARACTERISTICS 
CAN BE CONTROLLED ON A POINT-BY-POINT BASIS 



Figure 1 . Stylized Schematic of OTTO in Use 


FIGURE 2-SCHEMATIC OF LIQUID CRYSTAL CELL 

Figure 2 shows a schematic view of OTTO. The cell consists of a thin 
film photoconductor in contact with a thin layer of liquid crystal. Typically, 
these layers are 10-20 p thick. The photoconductors and liquid crystal 
layers are sandwiched between transparent electrodes across which a dc volt- 
age is impressed. The entire sandwich of layers is enclosed in glass. In 
order to understand the operation of the device it is necessary to consider the 
electro-optical character of liquid crystals. 

FIGURE 3-LIQUID CRYSTAL STRUCTURE SCHEMATIC 

Figure 3 illustrates the structure of the three basic types of liquid 
crystal. 3 in the nematic type, the cigar shaped liquid crystal molecules have 


104 



APPLIED 

VOLTAGE 



Figure 2. Schematic of Liquid Crystal Cell 
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a single axis along which they prefer to align. In the other two directions, 
there is no such preferred direction of alignment. In the case of cholesteric 
liquid crystals there is a second level of alignment. This type of liquid crystal 
organizes itself in layers. Within each layer there is nematic-type alignment. 
The nematic axis rotates from layer to layer thereby creating the spiral or 
helical structure that characterizes the cholesteric liquid crystal. The smectic 
liquid crystal is still more ordered in that the molecules form layers and all 
layers are aligned with respect to each other. In the OTTO device, we use 
nematic liquid crystals and we operate them in the dynamic scattering mode. 
The phenomenon of dynamic scattering is illustrated in Figure 4.^’ ^ 
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( c ) Choi es te ri c liquid c ry s tal s 
Figure 3. Liquid Crystal Structure Schematic 
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FIGURE 4-DYNAMIC SCATTERING 


Basically, dynamic scattering is characterized by the formation of small 
(about 1 \i) domains of aligned liquid crystal material that are in turbulent motion 
with respect to one another. These domains scatter light and therefore can be 
used to modulate a laser beam. The domains arise as shown, in Fig. 4. A dc 
voltage applied across the liquid crystal layer aligns the liquid crystal mole- 
cules. As the voltage is increased current stai’ts to flow due to the presence 
of impurities in the liquid crystal. At sufficiently high voltages, the charge 
carriers disrupt the aligned liquid crystal into the scattering domains that 
characterize dynamic scattering. Due to the strength of the elastic bonding 
between the liquid crystal molecules there exists a threshold voltage (on the 
order of ten volts) below which no scattering occurs. With those few remarks 
as background, it is now possible to explain the operation of the OTTO. 

Referring to Fig, 2, we can explain the principle of operation of the cell 
in the following way. The photoconductor is fabricated so that its impedance 
in the dark state is substantially higher than that of the liquid crystal. Thus, 
when the photoconductor is not illuminated it accepts most of the voltage drop 
thereby maintaining the liquid crystal below its threshold for dynamic scatter- 
ing. When light falls on the photoconductor, its impedance drops well below 
that of the liquid crystal and the voltage sv/itches over to the liquid crystal to 
exceed the threshold for dynamic scattering. This process occurs locally. 

The transverse impedance both of the liquid crystal and of the photoconductor 
are so high that there is virtually no loss of resolution due to transverse 
spreading of either the photocurrent or of the turbulent scattering. As a result, 




b. ABOVE THRESHOLD 


Figure 4. Dynamic Scattering 
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the resolution of the excitation process is approximately equal to the thickness 
of the liquid crystal layer. Since the liquid crystal layer is approximately as 
thin as typical photographic emulsions, these cells have photographic quality 
resolution. Thus, if we excite the photoconductor with an image— i.e. , a 2-D 
distribution of light intensity variation— the cell will transform this light into 
an identical scattering image which can then be picked up with a second beam 
of light. In the case of the OTTO, the photoconductor is excited with nonco- 
herent light and the scattering image is sensed with spatially coherent laser 
light. 

FIGURE 5- PRINCIPLES OF OPERATION OF THE OTTO 

Figure 5 shows how this process is carried out without destroying the 
spatial coherence of the laser light. 6 Consider two pencils of rays of laser 
light— one that strikes a portion of the cell that is in its scattering state and one 
that strikes the cell at a location where it is not scattering. If we place a lens 
and aperture each a focal length apart, then the undisturbed light passes through 
the aperture and can be reimaged on the output plane, whereas the scattered 
light is recollimated by the first lens and is blocked from appearing at the out- 
put plane. The point to note is that apart from a noise background which 
primarily affects contrast, all of the scattered light is blocked from the output 
plane. Only unscattered light reaches the output plane to form the image, 
therebjr insuring that the output is fully coherent. 


f — — f — — f — — f - — *| 



CLV 

Imaging optics 

Figure 5. Principles of Operation of the OTTO 



FIGURE 6-SYSTEM SCHEMATIC 


Figure 6 shows a schematic of the system that we use to operate the cell. 
The horizontal portion of this system is the optical data processor. It consists 
of a laser, a beam spreader, a recollimating lens, a 90 Hz light chopper (to 
avoid measuring light levels at dc), the OTTO, and the optical system for 
forming the amplitude image from the scattering image. Spatial filters for 
data processing would be placed just to the right of the blocking aperture. The 
vertical leg contains the v/rite beam. It is illuminated with noncoherent light 
that is focused onto the photoconductor of the OTTO via the pellicle beam- 
splitter. 

FIGURE 7-PHOTO OF TEST SETUP 

Figure 7 shows a photo of the experimental setup. As could be seen from 
the schematic (Fig. 6) the laser light passes through the photoconductor. The 
reason it doesn't "write up" the photoconductor on its way to the output plane 
can be seen in Fig. 8. 


LIQUID CRYSTAL 



Figure 6, Schematic Diagram of Liquid Crystal Light Valve Optical Processor 
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Figure 7. Test Setup 




FIGURE 8— CDS SENSITIVITY CHARACTERISTICS 


This figure shows the absorption characteristics of CdS, the photoconductor 
that is used in the OTTO, as a function of wavelength. The heavy curve shows 
this characteristic in the absence of liquid crystal and the dashed curve shows 
it in the presence of liquid crystal. Consider first the dashed curve. In the 
presence of liquid crystal, the sensitivity of the CdS to HeNe laser light is 
down by two orders of magnitude compared with that at the peak sensitivity of 
the CdS at fiftOO A in the green. This is the reason that we can pass the laser 
light through the photoconductor without exciting the cell. Figure 8 illustrates 
a second very important characteristic of CdS in this application; viz, the 
presence of the liquid crystal significantly alters the sensitivity characteristics 
of the CdS. The spectral response of the CdS in the presence of liquid crystal 
is lifted in the blue and falls off more rapidly in the red. Both effects enhance 
the performance of the device. In the presence of liquid crystal the CdS is 
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Figure 8. <CDS Sensitivity Characteristics 


substantially more sensitive to the noncoherent write beam and substantially 
less sensitive to the coherent read beam. This effect is due to the formation 
of a diode -like, depletion region in the vicinity of the interface between the 
liquid crystal and the CdS. This depletion region also increases the dark 
impedance of the CdS. As a result, it makes the impedance match between the 
CdS and the liquid crystal more nearly ideal than would be thought possible 
simply on the basis of the bulk resistivity of the CdS alone. 
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FIGURE 9-OTTO CHARACTERISTICS 

Figure 9 summarizes the operating characteristics of the OTTO. It is a 
reasonably high resolution, very sensitive, somewhat slow, high efficiency 
and low operating voltage device. The ac voltage is applied to speed up the 
cell and to promote uniformity of scattering. 

FIGURE 10— OUTPUT PLANE PHOTO 

Figure 10 shows some imagery recorded with the OTTO. This is a photo- 
graph of the output plane image for the case of a checkerboard input image. 
This was recorded in laser light. The input image was created with non- 
coherent light from a tungsten lamp with the system shown in Fig. 7. 

FIGURE 11 -BLOWN UP VERSION OF OUTPUT PLANE PHOTO 

Figure 11 shows a blown up version of the image shown in Fig. 10. The 
input image had 90 squares across a one-inch field. Note the sharp edges of 
the dark areas. This edge detail is obtained by virtue of the high resolution 
capability of the OTTO. 


RESOLUTION 

70 1 i nes per mm 

1 SENSITIVITY 

1 yW/cm^ @ 510 nm 

CONTRAST RATIO 

500: 1 

EXCITATION TIME 

100 msec 

DECAY TIME 

100 msec 

COHERENT LIGHT 


THROUGHPUT 

95 % 

OPERATING VOLTAGE 

40 VDC plus 
27 VRMS t 20 KHz 


Figure 9. OTTO's Performance Characteristics 
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Figure 10. Output Plane 


Figure 1 1 . Blown Up Version of 
Output Plane 
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THE SYNTHESIS OF COMPLEX SPATIAL FILTERS 
FOR COHERENT OPTICAL DATA PROCESSING 

Sing H. Lee, Department of Electrical Engineering 
Camegie-Mellon University, Pittsburgh, Pennsylvania 15213 

INTRODUCTION N 7 3 ra 18693 

The fundamental advantages of coherent optical processors are their in- 
formation processing capacities and speed. Comparing with their incoherent 
counterparts coherent processors are also more flexible in that they can per- 
form "complex" spatial filtering. 

The key elements in coherent optical processors are the spatial filters. 

Therefore, in order to realize the full potential of coherent processors much 
work should be devoted to the synthesis of and the production techniques for 
them. In this paper, some recent work on filter synthesis will be summarized, 
and the implications in optical computation, image processing (enhancement), 
image restoration (deblurring) and image detection examined. In addition, some 
advances on the production techniques for the spatial filters will be briefly re- 
viewed. This would include the extension of Vander Lugt's filters to the 
multiply-exposed filters, the binary filters, ® - ^ the phase only filters,® 
and the introduction of nonlinear filters. ^ 

FILTERING FUNCTIONS 

For Optical Computations 

On filter synthesis we shall first discuss filters for optical computations. 

The most fundamental linear operations in optical computations are translation, 
addition, subtraction, differentiation and integration. The filtering functions, 
which can perform these computation operations, are surprisingly simple, and 
consist of no more than a proper collection of sinusoidal gratings. 

Optical Translation . Given an optical pattern function in the input plane of 
a coherent processor, its image which appears in the first diffraction order can 
be translated with respect to the optical axis in the output plane by controlling 
the grating spacing of a (sinusoidal) diffraction grating set in the filtering plane 
(Fig. 1). The higher the frequency of the grating is, the farther away from the 
optical axis the diffracted images will be. 
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Figure 2. Optical Addition and Subtraction 


Optical Addition and Subtraction! . Consider the shifting in position of a 
pattern function in the input plane (Fig. 2). Since a coherent optical system is 
space-invariant, its image in the output plane will be shifted in position also. 
That is, the image of the letter "S" in the input plane, which is shifted upward 
with respect to the optical axis, will appear shifted downward with respect to 
the optical axis in the output plane. Similarly, the image of the letters "OA", 
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which is shifted downward in the input, will appear shifted upward in the output. 
Notice that in the central region of the output plane the image of the letter "S" 
overlaps with the image of the letter "OA", if the frequency of the grating in 
the filtering plane is chosen properly to correspond to the separation between 
the two input pattern functions. Furthermore, in the overlapping region, the 
two images will interfere constructively (or add) if the grating function is a co- 
sinusoidal one with maximum transmittance at the optical axis as illustrated in 
Fig. :t (a) . On the other hand, the two images will interfere destructively (or 
subtract) if the grating function is a sinusoidal one with minimum transmittance 
at the optical axis (Fig. 3(b)). 

Optical Differentiation and Integration . To differentiate a pattern function, 
the method of finite differences can be applied. According to the concept of 
finite differences, the differentiation of a pattern function can be arbitrarily 
closely approximated by taking the difference between two images, one of which 
is shifted with respect to the other by an arbitrary small distance. To generate 
the two images in the output from one pattern function in the input, two gratings 



Figure 3. (a) Complex Amplitude Addition. The pattern function 
"IT" is added onto that of "CIT" in the central region of 
the output plane; (b) Complex Amplitude Subtraction. 

The pattern function "IT" is subtracted from that of "CIT" 
in the central region of the output plane. 
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in the filtering plane will lie necessary. I'lie small displacement of one image 
from the other can then be obtained by choosing the two gratings of slightly dif- 
ferent frequencies. Pin* difference (or the subtraction) between the two images 
will then be ensured, when one of the grating functions has maximum trans- 
mittance at the optical axis and the other minimum transmittance at the optical 
axis. Figure 4(b) illustrates the differentiation along the horizontal direction 
of the pattern function of Figure 4(a). 

For more complex or higher order differentiations (Fig. f>), more gratings 
will hi- needed in the finite difference approach. Hut, gratings are usually 
fairly easy to make. 

To integrate a pattern function, if one can accept the fact that the integra- 
tion of a function is simply the sum of many of its images shifted by various 
amounts, it becomes obvious that a collection of gratings of various spatial 
frequencies will also provide the desired integration result in the output, To 
summarize, filtering functions consisting of composite gratings can be utilized 
to perform a number of fundamental computational operations. Once these 
composite gratings are prepared they can operate on any new input pattern 



Figure 4a. The Object Pattern for Figure 4b. Experimental Result for <H Ox 
Di fferentiations 


llt< 



information. Hence, they are well suited to the role that a coherent processor 
can play as a computer, 

A different perspective on performing optical computations by composite 
gratings can be visualized when the Fourier transform of a sinusoidal grating 
is recognized as the sum of two delta functions. Optical processing with such 
a grating performs in effect the correlations of the two delta functions and the 
input pattern. Only one of the correlations normally contributes to the desired 
processed result. 1’his is evident from the earlier discussions and examples, 
which show that onl\ one diffraction order from each grating contributes to the 
processed result. Therefore, when processing with composite gratings, the 
correlation functions of the input pattern and one of the delta functions associ- 
ated with each "component" grating (in the composite grating) interfere among 
one another and give rise to the processed output. Mathematical! v. the com- 
putational operations can be expressed in the following forms. 

Translation K x+a.y ) = f(x,y ) * 5< x-a.y ) (1) 

Addition f(x.y) + g(\,y) = l'< \-a.y) * 5(x+a.y ) + g(x+a.y > * 5(x- a.y ) (2a) 



Figure 5a. Experimental Result for 
ft f fix + f)f/3y 


Figure 5b. Experimental Result for 

3 2 f/Dx 2 + 3 2 f/0y 2 
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( 2 b) 


Subtraction f(x,y) - g(x,y) = f(x-a,y) * 5(x+a,y) + g(x+a,y) * 6 (x-a,y) e in 


Differentiations 


0 f 

— = lim 

9x h-0 


f(x,y) * g 


j(x,y)j 


g 1 (x,y) = 5(x+h,y)- 5(x,y); 


0 f 

— = lim 

0 y h-o 


7 - f(x,y)*g 2 (x 
h 


4 


g 2 (x,y) = 5(x,y+h) - 5(x,y); 


(3a) 


(3b) 


df 3f 
— + — = lim 

9x d y h-o 


h 


f(x,y) * g 3 (x,y) 


g 3 (x,y) - 5(x+h,y) + 6 (x,y+h) - 25(x,v) 


(3c) 


0 2 f 0 2 f 
+ lim 

0X 2 0X 2 


h-0 


f(x,y) * g 4 (x,y) 


g 4 (x,y) = 5(x+h,y) + 5(x-h,y) + 5(x,y-h) + S(x,y+h) - 45(x,y) 


0 2 f 

0 x 0 y 


= lim 


h-0 


r 1 

— f(x,y) * g 5 (x 
h 2 


4 


g s (x,y) = 5 (x+h,y+h) + 5(x,y) - S(x,y+h) - 5(x+h,y) 
Integrations: jf x = f(x,y) * u(x,0) 

f K f y f(£,T?)d£ dr? = f(x,y) * u(x,y) 

jfr«i J — 00 


(4) 


(5) 

( 6 a) 

( 6 b) 


where the symbol * denotes the correlation operation, u(x, 0 ) is a slit function 
and u(x,y) a step function. 
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For Image Enhancement 


In image enhancement there are two situations often encountered. One 
situation requires an image to be smoothed in order to suppress noise that may 
be presented in it. The other requires an image to be sharpened in order to 
extract contours and edge details. As in the last section and the sections that 
follow, we again concentrate our attentions on the methods that can be imple- 
mented by a coherent processor. 

Smoothing 

a. Band-pass Spatial Frequency Filtering . Lowpass and bandpass spatial 
filtering are the simplest methods in suppressing noise, when the 
spatial frequency characteristics of noise is substantially different 
from that of the picture. The filtering functions in this kind of filtering, 
as is well-known, consists of stops (or zeroes) in those frequency 
regions, where noise gives most contribution. 

b. Averaging over a Neighborhood . One can also smooth a picture by 
simply replacing its value at each point by the average of the values 
over a neighborhood of the point. This method of smoothing is, there- 
fore, just the correlation of the picture with an aperture of an area 
over which the values are averaged, and can be considered as a proc- 
essing operation similar to that described by integration. 

c. Averaging of Multiple Images . Another method for improving the 
signal-to-noise ratio of a picture is by averaging a set of independent, 
noisy images of a picture. In principle the averaging can be done by 
optical summation as described above, though in practice the constraint 
on the quality of the optical system should be considered in using this 
method. 

Sharpening . Just as one can smooth a picture in a variety of ways, so there 
are many ways to sharpen a picture. For example, highpass spatifll filtering 
or some sort of differentiation operation (such as the gradient and the Laplilvlan 
operations) can enhance a picture by extracting contours and edge details. Thus, 
the filtering functions for differentiation operations described earlier for com- 
putational purpose are also useful here in image enhancement. 

For Image Restoration 

9— T 2 

There are many methods to restore a degraded image. But, in this 
section we shall continue our attempt with composite gratings to illustrate that 
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when they are properly synthesized, they can play an important role, as they 
have in optical computations and image enhancement. (Note once again that 
processing with a filter of composite gratings is the same as processing the 
input pattern function (in this case a degraded image) by correlating it with a 
collection of delta functions. ) 


To begin, the simple case of an image degraded by N multiple exposures 
will be considered. Between exposures the object illuminance is displaced by 
a distance A l’elative to the recording medium. 


In this case the degraded image gj can be expressed as 


where 


gj = f * hj, 

h i = E a n 5(x-nA), 

n=0 


(7) 

( 8 ) 


and a n corresponds to the intensity of the n^ 1 exposure. Our objective is to 
synthesize a processing function, p , consisting of a set of delta functions so 
that when this processing function correlates with gj(x,y), the image function 
f(x,y) can be retrieved within a certain area of interest. We let 

M 

p(x,y) = X] b m 5 (x~mA), (9) 

m=0 

where b m may take on any values, positive, negative, or even zero. Correlating 
g, (x,y) with p(x,y) gives 


g!(x,y) * p = f * hj * p. 

N M 

= f*E E 

m=0 m=0 

The correlation (hj * p) is illustrated in Figure 6 where we have in the m tb 
row the m^ 1 delta ftinction associated with p(x,y) correlating with the blurring 
function hj. It is clear from this figure that the condition for (g 2 * p) be equal 
to f(x,y) within the region |x|<(q/2)A requires 


a n b m [x~ (n+m)AJ . 


( 10 ) 
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a o b o 

a l b 0 


a 2 b 0 


+ a 0 b 
+ a^^- a Q b. 


= 1 
- 0 
= 0 


a q-l b 0 + a q-2 b l + 


• + a 0 b q-l =0 ‘ 


This set of conditions or equations allow us to determine the processing function 
p(x,y) , or equivalently the coefficients b m . Writing the set of equations into 
the matrix form, we have 


where 


A b = c, 

'v <v 



0 

0 


a q-2 

a q-3 • 

• - - a o 

0 

Vl 

a q-2 • 

, . . .aj 

a o 


a o 0 

a i a o 0 


( 11 ) 


b o 


1 

b l 


0 


and c = 



A/ 


b q-2 


0 

b q-l 


0 


Since the matrices A and the vector c are known, the vector b can easily be 

A/ Ay i A/ ■ 

determined by matrix inversion, b = A c. As an example, consider an image 


123 
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j a o b 2 | a | b 2 j a 2 b| 

* a o b i f a i b i fa 2 b it° 3 b , 

I 1 i | ° '• 


I 


:°N b 3. 


' a 3 b 2. 




" a _ b 2 t 1 

<J'2 4. , 


°N b 2. 




a o b o‘ 


a, b 0 


> I 

a 2 b o ' 1q 3 b o g 


V». 


a N b 0 




Figure 6. The Correlation of h 1 (x,y) and p(x,y) 

degraded due to the malfunctioning of a camera such that the recorded image 
has four exposures. Betweeh exposures there is a relative displacement A 
between the object illuminance and the film. If all four exposures are identical, 
hj(x,y) then becomes the function as shown in Figure 7a. Further, if f(x,y) is 
to be restored from g(x,y) at least within a region of width 4A , the vector c 
becomes 


c = 

'V 


(12a) 


From our matrix formulation the vector b is found to be 


b = A 1 c = 

'V W 'V 


1 

-1 

0 

0 


(12b) 
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Figure 7. (a) The PSF for an Image of Four Exposures; 

(b) The Processing Function Synthesized from the Matrix Formulation 
for the Restoration of an Image of Dimension up to 4A Along 

the x-Direction; 

(c) The Result of Correlating h 1 with p t ; 

(d) The Processed Result of g 1 (x,y) by p., (x,y). 



The solution for b leads to the processing function Pj(x,y) as shown in Figure 
7b, It is clear that an image of dimension up to 4A along the blurred direction 
has been restored as intended (Fig. 7c, 7d). Figure 8 shows one of our experi- 
mental results obtained using the processing function Pj(x,y). If an image of 
dimension larger than 4A , but smaller than 8A , is to be retrieved from a rec- 
ord degraded by the same PSF as in the last example, similar analysis shows 
that c now becomes 


0 

0 

0 

0 

c 

0 

0 


(13a) 


The solution for b is 


-1 

0 

0 

1 

-1 

0 

0 


(13b) 


The processing function Pi(x,y) corresponding to this solution of b is as illus- 
trated in Figure 9. Comparison between Figures 7 and 9 indicates that the 
addition of another pair of delta functions (one positive, the other negative) to 
p'j(x,y) helps to move the ghost image away such that a f(x,y) of relatively 
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Figure 8. Image Restoration, (a) An image of a portion of the Moon degraded 
by malfunctioning of the camera shutter; (b) A photo of the output 
obtained from the restoration scheme described in text. 

This page is reproduced at the 
hac k of the report hy a different 
reproduction method to provide 
better detail. 
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larger dimensions can be recovered. In extension, by adding more pairs of 
delta functions one would be able to restore a f(x,y) of even larger dimensions. 
It should be pointed out that since the ‘coefficients a n , b in our approach are 
not restricted to equal magnitudes, the multiple exposures in g(x,y) need not 
be equal for restoration to take place. 

Next, let us consider images degraded by linear smear. The PSF for 
linear smear degradation, h ? , is a rectangular function of width A' (Fig. 10a). 
Since a rectangular function is a limiting case of h in Figures 7 and 9 when A 
is allowed to approach c, an arbitrary small distance, but (N- 1)A is kept 
constant at A' by increasing N, the solution for the processing function p 2 (x,y) 
can be deduced from the solution Pj(x,y) obtained earlier for the multiply- 
exposed type of degradation. Hence, p,(x.y) of Figure 10b is deduced from 
p,(x,y) and p,(x,y) of Figure lOe from p',(x,y) for the 1 ’estoration of linearly 
smeared images of dimensions up to A' and 2 A' respectively. As is obvious 
p 2 (x,y) and pi (x,y) are merely collections of doublets. In extension, one can 
also show that the restoration function for linear acceleration and deceleration 
type of degradation with a blurring function of a triangle is the differentiation 
of doublets. 

For Image. Detection 

Image detection with coherent light has most often based on matched filter- 
ing. 14-lh By way of definition, a linear filter is said to be matched to the 
particular signal i'(x,y) if its impulse response g(x,y) is given by the complex 
conjugate of f(x,y>, , 

g(x,y) = f*(-x, -y), (14) 

Matched filtei’ing works very well, when it is used to detect an image among 
many which bear little resemblance. But, for similar images, different types 
of filtering with better discriminating capabilities appear to be needed. Two 
simple approaches have been studied with this intention in mind. One makes 
use of the gradient correlation, 2 and the other make use of the subtraction- 
correlation. 3 

The Gradient Correlation . The gradient correlation recognizes or detect 
the input signal by its perimeter or outline, in contrast to the correlation ol 
the signals themselves recognizes it by area. Mathematically the gradient 
correlation is represented by 

s(x,y)= / / grad f(x\y')grad fnx'“X,y'-y) dx'dy' 
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Figure 10. (a) Linear-Smear Degradation; (b) The Processing Function for the 
Restoration of an Image of Dimension up to A' (c) The Result of 


Correlating h 2 by P 2 ; (d) The Processed Result of g 2 (x,y) by p 2 (x,y); 
(e)“(g) same as (b)-(d) except that the processing function p 2 (x,y) 
is synthesized for the restoration of an image of dimension up to 2A'. 
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(15) 


(9 2 5/9x 2 ) + 


/ 9 2 8 ' 


v9y 2 


Expressing the Laplacian operation on the delta function in terms of a differ- 
ence equation as was done earlier on optical computations, 


(9 2 5/9x 2 ) + (9 2 5/9y 2 ) - lim (— \ (5(x+h,y) + 5(x-h,y) 

h-*0 l\ h2 / 

+ 5(x,y+h) + 5(x,y-h) - 46(x,y)] 

we have 


(16) 


s(x,y) = lim ( — ) [f(x+h,y) + f(x-h,y) + f(x,y+h) 

h-*0 l\ h2 / 

+ f(x,y-h) - 4f(x,y)]| * f(x,y). 

The filtering function which achieve gradient correlation is 


(17) 


G(u,v) = lim — j f exp(— i27ru -hj^ ) + exp(i27ru*h 1 ) (18) 

+ exp(-i2irvh 2 ) + exp(i2Trv*h 2 ) - 4] | F*(u,v) 

Figure 11(a) illustrates the impulse response of such a filter which has been 
synthesized for the character T. It is to be compared with the impulse response 
of a matched filter (Fig. lib). Figure 12 shows the response of the two filters 
to the letters C, I, T. Clearly, the gradient con-elation filter gives better de- 
tection discrimination between the similar images of I and T, when the intensity 
profiles across the central part of the correlation functions are examined on 
an oscilloscope display. 

The Subtraction Correlation . The concept of employing the subtraction 
operation to extract a desired feature which characterizes an image and utiliz- 
ing the correlation operation to detect it is quite straightforward. To implement- 
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Figure 12a. Correlation for the Input Letters CIT Using a 
Gradient Correlation Filter of the Letter T 
Synthesized by a Multiple Exposure Technique 


Figure 12b. Correlation for t he Input Letters CIT Using 
a Matched Filter of the Letter T 


Figure 1 la. Impulse Response of a 
Gradient Correlation 
Filter for the Letter T 
Synthesized by a Multi- 
ple Exposure Technique 


Letter T 


Figure lib. Impulse Response of a 
Matched Filter for th« 





Figure 12c. Intensity Profiles Across the Center Lines of the 
Correlation Patterns for Input Letters CIT and (a) 
the Gradient Correlation Filter (b) The Matched 
Fi Iter of the Letter T 


it optically in a single-stage coherent processor, one feasible and convenient 
method is to display both image functions simultaneously in the input plane, one 
centered at say bj distance from the optical axis, the other at h 2 from the opti- 
cal axis. A spatial filter bearing the information of 

t(v) = Fj*(u,v) (cxp(i2irvhj >- exp(i27rv*h 2 )| (19) 

will be needed, where I *(u,v) stands for the complex conjugate of the Fourier 
transform of the feature to be detected. For example, we wish to distinguish 
R from IV The feature to be extracted from this letter pair is then the tilted 
stroke. Figures 13 and 14 illustrate the experimental result of our combined 
subtraction and correlation procedure when the inputs correspond to R and l 3 , 
and l 3 and P respectively. The presence and absence of the feature of interest 
in the central region of the output plane is obvious and can be easily detected. 
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Figure 13. The Output of a Coherent 
System with a Subtraction- 
Correlation Filter. Patterns 
from left to right are 
<f 2 f 3 )*f,, and f 3 *f, 

where stands for the 
tilted stroke, f 2 the char- 
acter R, and f 3 the char- 
acter P . 


Figure 14. The Output of a Coherent 
System with a Subtraction- 
Correlation Filter. Patterns 
from left to right are f 2 * f 1 , 
(f 2 - f 3 )* f ,, and f 3 *f,, 
where f 1 stands for the 
tilted stroke and both f ? 
and f 3 are the cha r acter P. 


FILTER PRODUCTION 
The Multiply- Exposed Filter 

Normally, Vander I.ugt's interferometric method is very convenient to 
employ in complex filter production (Fig. 15). However, on a number of occa- 
sions as we have seen earlier that composite filtering functions consisting of 
a collection of gratings or a sum of several complex functions do arise. For 
these occasions there is a need to extend Vander I.ugt's technique to include 
multiple exposures. As an example, the gradient correlation filter of Eq. (18) 
is to be produced. Since the filtering function for gradient correlation consists 
of five terms, five exposures will be necessary. In these exposures the same 
reference wave is to interfere with the Fourier transforms of f(x.y) , but before 
each subsequent exposure is made, the position of f(x,y) will be shifted slightly 
by the amount h. Note also that one of the exposures should be four times 
stronger or longer than the others, and the negative sign can be introduced by 
shifting the film plate by half a fringe relative to its position for the other ex- 
posures. The impulse response of a gradient correlation filter for the letter T 
has been shown earlier in Figure 11(a). 

The Binary Filter 

Another method of producing a complex filter employs the digital computer 
to provide a binary filter (Fig. 1(5). Binary filters work on the optical diffrac- 
tion phenomena of slits. The important procedures involved here are (1) to 
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IMPULSE 

RESPONSE 


Figure 15. Two Interferometric Systems for Producing the 

Frequency Plane Mask — the Vander Lugt Filter 

_ sin 0 

OL- — 

sample the filtering function at periodic lattice points, (2) determine the dimen- 
sions of the transparent slits so that the amount of light passing through them 
are proportional to the magnitudes of amplitude transmittance called for by the 
filtering functions at those points sampled, (3) shift the position of each slit by 
an amount proportional to the phase angle of the filtering function at that point, 
(4) photoreduce the plotter output from the computer. 

In Lohmann's method^’ ® the phase angles of the filtering function at the 
sampled points are quantized. Since phase quantization introduces noise in 
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The Non-Linear Filter 


So far most optical complex filterings have been performed only with filters 
produced by any one of the three techniques just described. Recently we have 
been investigating the possibility of performing filtering without having to go 
through the process of preparing the filters. One promising scheme that we 
have found 17 is shown in Figure 17. In the scheme always the same filtering 
element is supposed to be used, while different filtering operations are to be 
performed. The processing functions, g, must then be placed also in the input 
plane together with the images to be processed, f. In order for the Fourier 
transforms of the two functions, f and g to interact and multiply with each 
other, so as to yield the result of convolution, correlation or other linear oper- 
ations, we need to choose in the filtering plane a nonlinear element with ampli- 
tude transmittance characteristic proportional to the intensity incident on it. 
Further, in order for the convolution and correlation functions to be physically 
separated from other undesired outputs in the output plane, we introduce a 
reference delta function in the input plane. Preliminary experimental results 
using this scheme is very encouraging. But, we can not elaborate anymore on 
it now. It is planned, however, that a paper with details of analysis and more 
experimental result on this scheme will be submitted to the OSA meeting come 
this fall. 
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DESIGN CONCEPTS FOR AN ON-BOARD COHERENT 
OPTICAL IMAGE PROCESSOR 
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INTRODUCTION N?3" 18694 

Optical data processing techniques have been known for the past one hun- 
dred years and are the oldest documented techniques for information! process- 
ing. The research and development work of the past fifteen years has given 
a new impetus to this field. A number of techniques such as those for diffrac- 
tion pattern analysis, amplitude and complex spatial filtering, matched filter- 
ing and auto and cross correlation are now fairly well understood. The inherent 
two dimensional nature of the optical system makes them superior to electronic 
counterparts which take the information sequentially or on bit by bit basis, and 
are inherently one dimensional. 

In particular, the need for parallel image data processing techniques for 
analysis, synthesis and extraction of information arises from the fact that an 
enormous amount of pictorial type data about the earth's features will be 
acquired by NASA's future Earth Resources satellites. To make good use of 
this data for earth's agricultural, geological, hydrological, geographical, 
oceanographical and environmental conditions, an efficient and timely processing 
of data is necessary. The use of coherent optical techniques is one of many 
techniques that will be an aid in handling this problem. On-board spacecraft 
image data processing systems will be very useful because they will allow 
transmission of processed data rather than raw data, resulting in an enormous 
saving of transmitter bandwidth. 

HISTORICAL DEVELOPMENT OF OPTICAL DATA 
PROCESSING TECHNIQUES 

In 1873, Abbe 1 in his classical paper reported a number of experiments 
in which he manipulated the diffraction pattern produced by specimens in a 
microscope system. The main aim of his work was to improve the resolution 
in microscopes. These experiments were performed to seek the explanation 
of the observation that, "a large angular aperture results in a more brilliant 
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image of higher resolution than small angular apertures— even though the cone 
of the light emanating from the light source fills only a very small portion of 
the aperture, leaving the remainder seemingly unused as dark space". After 
a number of experiments he discovered that the influence of angular aperture 
is due to diffraction phenomenon. Porter^ following Abbe's work and realiz- 
ing that a complete mathematical development has never been published, pre- 
sented a paper in 1905, in which he reported a number of band pass type 
spatial filtering experiments and treated the phenomenon by means of a simple 
application of the Fourier Theorem. Zernike 8 in 1932 proposed a technique for 
converting spatial phase modulation to spatial intensity modulation. This tech- 
nique was also based on Abbe's theory and is known as the, "Phase Contrast" 
technique. Gabor^ in 1948 presented the idea of wave-front reconstruction. 

He described the theory and performed experiments to demonstrate that a 
recox’ded diffraction image could be reconstructed to a normal image. Due to 
the lack of suitable coherent light sources at that time, the full potential of this 
technique was not realized. This was a unique two step method of optical 
imagery. In 1952, Elias®, et al. published a paper in which they pointed out 
that some optical systems are analogous to electronic filter circuits. Elias^ 
in another paper published in 1953 illustrated the application of a few mathe- 
matical techniques originally developed for electrical communication theory to 
optical problems. 

The work of Marechal and Croce 1 first published in 1953 was mainly con- 
cerned with the improvement of photographs. By using coherent optical data 
processing techniques they were able to suppress a given periodicity in the 
image, improve the details and the contrast of the details and to remove the 
spherical aberration and astigmatism from a recox’ded image. A year later in 
1954 Cheatham and Kohlenberg presented a paper emphasizing that many optical 
systems perform a linear transformation of an object plane equivalent of the 
one dimensional time transformation performed by a linear electrical filter, 
many mathematical techniques developed for treating linear electrical filters 
can be used to study the optical systems. They also presented the methods of 
synthesizing optical filters. 

O'Neill 8 in 1956 published a comprehensive paper in which detailed 
theoretical and experimental results were presented that demonstrate the anal- 
ogy between optical and electrical filtering. In 1959, Cutrona^, et al. published 
a paper in which they described the configurations and the properties of multi- 
channel coherent optical signal processing system. It was pointed out that the 
two dimensional nature of optical systems makes them very attractive to sup- 
plement or even to replace complex electronics systems for performing a 
number of linear mathematical operations. Van Heerden^ in a paper pub- 
lished in 1963 presented the theory of an "Intensity Filter" for the focal plane 
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of an imaging system. Projected capabilities of this filter in a recognition 
system were also explored. 

Slabodin in his communication also published in 1963 presented an optical 
technique for correlating electronic signals in realtime. 

In 1964, 12 Vander Lugt introduced a unique technique for synthesizing 
complex spatial filters. The salient feature of this technique was the transfor- 
mation of phase and amplitude information into intensity information so that 
they can be recorded on a photographic plate. Modified Mach-Zehnder and 
Rayleigh interferometric arrangements were used for the synthesis of these 
filters. The introduction of this technique was largely responsible for a re- 
evaluation of the coherent optical spatial filtering techniques for recognition 
and detection purposes. 

The works of Stroke- 1 - 2 3 4 5 and co-workers and Lohmannl^ and co-workers 
has also benefited the various aspects of optical data processing. 

THE COHERENT OPTICAL DATA PROCESSING SYSTEM AND 
EXPERIMENTAL RESULTS 

Figure 1 illustrates the conventional configuration of an optical system 
normally used for coherent optical data processing. Light from a He-Ne laser 
radiating at a wavelength of 632. 8 nm is brought to a point focus by a micro- 
scope objective. A pinhole is placed in this focal plane to eliminate laser beam 

5 7 9 



1. He-Ne GAS LASER 6. FOURIER TRANSFORMING LENS 

2. MICROSCOPIC LENS 7. FOURIER TRANSFORM PLANE 

3. PIN HOLE 8. IMAGE RECONSTRUCTING LENS 

4. COLLIMATING LENS 9. IMAGE RECONSTRUCTION PLANE 

5. INPUT PLANE 

Figure 1 . Conventional Configuration of a Coherent Optical System 
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noise and to exclude stray light. Another lens is placed a focal length away 
from a converging lens. This lens in its focal plane produces the Fraunhofer 
diffraction pattern of the input information in two dimensions. This diffraction 
pattern has all the characteristics of a two-dimensional Fourier transform of 
the light distribution in the input information. If an arbitrary function is intro- 
duced in the input plane, a two-dimensional spectrum of that function is obtained 
in the Fourier transform plane. If a complex spatial filter is placed in the 
Fourier transform plane, the system can be used for autocorrelation or cross- 
correlation. 

This optical system works well in the laboratory environment. Aboard 
spacecraft, however, this system has two main drawbacks. The first is the 
low efficiency and bulkiness of the gas laser, and the second is the lens system. 
Disadvantages of lens systems are front surface reflections, which are unavoid- 
able, and the requirement for axial symmetry of the system (i.e. , the optical 
system with lenses cannot easily be folded). 

A solid-state GaAs laser diode has been used to overcome the first draw- 
back. This laser is at least ten thousand times better in efficiency and probably 
the same order of magnitude smaller in overall volume. Furthermore, since 
a GaAs laser can be treated as a point source, the system does not require the 
microscopic lens and pinhole spatial filter minimizing optical alignment 
problems. 

To overcome the second problem, paraboloidal mirror segments are used. 
It has been found that paraboloidal mirror segments not only overcome the 
axial symmetry problems but have a number of significant advantages over 
lenses. Let us start by considering the problems associated with lenses. 

The quality and resolution of Fourier transform functions in the focal plane 
of a lens is very much affected by the lens aberrations and the optical refract- 
ing material. The three types of aberration that are primarily responsible for 
the degradation of the Fourier transforms are spherical aberration, astigma- 
tism, and coma. 

Besides overcoming these aberrations, other critical problems with lenses 
consists of the choice of the optical material. For an ultrahigh-quality lens 
suitable for data processing, the optical material should be: 

(1) homogeneous to a very high degree. (It is important that the refrac- 
tive index of the lens be constant throughout. ) 

(2) free from thermal and mechanical strains. 
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(3) optically isotropic (i.e. , the index of refraction at any point must be 
constant regardless of the direction in which the radiation is passing 
the point). 

Therefore, for a good optical system with lenses, it is necessary to rectify 
all the aberrations and to insure the selection of proper optical material. Both 
of these requirements are extremely difficult and costly to satisfy. 

Let us now consider the paraboloidal mirror as a linear element in an 
optical data processing system. Referring to figure 2, light rays entering 
from the right parallel to the z axis are reflected from the mirror and inter- 
sect the z axis at the focal point f. The relationship between the incident and 
reflected scalar light fields at any point in the (x,y,z 0 ) plane can be described 
in terms of a transfer function, t(x,y,z 0 ). For mirrors (as well as for lenses) 
the transfer function is strictly a phase function, and we have 

t(x,y,z 0 ) = ej 0(x >y> z o) ^ 

= e jkA(x,y,z 0 ) 



Figure 2. Transfer Function for Parabolic Mirror 

\ 


145 


where 0(x,y,z o ) is the phase difference in radians between the incident and re- 
flected fields, k is the wave number 2ir/\ and A(x,y,z Q ) is the difference in 
pathlengths between the incident and reflected waves. 

Again referring to figure 2, we see that the total path difference is 


A(x,y,z 0 ) = 2(z 0 - Az) (2) 

(This assumes no ray bending to the left of the (x,y,z 0 ) plane, analogous to the 
thin-lens approximation in lens analysis.) 

Now from the equation for a paraboloid of revolution, 


Az = 
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x z + y- 
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Therefore, 


t(x,y,z Q ) = e j kA (Wo) 
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In the case of a spherical thin lens, if the paraxial approximation is made, i.e. , 
if we consider only light rays close to the lens axis, the transfer function is 
found to be 


t(x,y,z 0 ) = c [jknz ole 
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7 7 

+ y~ 
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(5) 


Here n is the index of refraction of the lens material, and z Q is the lens thick- 
ness along the optical axis. It is important to note that the focal length f of a 
lens is a function of the index of refraction of the lens material. 

Referring to the derivation of the transfer function, several points can be 
made for the parabolic mirror as a system element. First, since the focal 
length f of a mirror is not a function of index of refraction, the parabolic 
mirror system will not have the chromatic aberration that is inherent in the 
lens system. Secondly, all the light rays parallel to the z axis will intersect 
the point f for a parabolic mirror, while only the paraxial rays will intersect 
the focal point for a spherical lens. This means that the parabolic mirror is 
inherently free from spherical aberrations (while the lens, of course, is not). 
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The absence of paraxial approximation for mirrors has several practical 
advantages. One is that the aperture of the optical signal being processed can 
be larger for a parabolic mirror than for a lens of the same diameter. More 
important, off-axis segments of a parabolic mirror will have the same trans- 
forming properties as the original mirror. Thus one can construct a folded 
optical processing system. 

Two other aberrations also deserve some attention. Let us consider astig- 
matism. This aberration occurs when the incoming light rays make a large 
angle with the z axis of a mirror. This is not a serious drawback, however, 
since for most optical signal processing, the incoming light is a coherent plane 
wave whose rays are parallel to the z axis. Furthermore, all interelement 
light paths can be kept at very small angles with respect to the element axis. 
Astigmatism therefore can be made negligible in a parabolic processing system. 

The second aberration is coma, the aberration that occurs for light rays at 
small angles to the z axis. In experiments using parabolic mirrors as Fourier 
transforming elements, the effects of coma were not observable at the normal 
working angles. 

Figure 3 illustrates an optical data processing system using paraboloidal 
mirror segments. The coherent light source (1) is a p-n gallium arsenide laser 
diode with an effective radiating area of 0.0015 mm^. The laser operates at 
room temperature at a wavelength of 900 nm. It is placed in the focal plane of 
the paraboloidal mirror (3). The coherent light (2) from the laser diverges and 
typically forms a 30 angle. Since the coherent light source is almost a point 
source and since it is in the focal plane of the paraboloidal mirror (3), the light 
reflected from the paraboloidal mirror segment (4) is collimated. 

To perform a two-dimensional spectrum analysis, the information to be 
analyzed is produced in transparency form (either by taking a photograph or by 
other means). The transparency (6) is placed in the collimated beam (5), a 
focal length away from the paraboloidal mirror (7). The paraboloidal mirror 
segment ( 8) converts the information of the transparency into a diffraction 
pattern that has all the properties of a Fourier transform in the plane (10) cen- 
tered at the point ( 9) which is the focal point of the paraboloidal mirror (7). 

The information in the diffraction pattern can be converted into electrical signals 
by placing an appropriate photon detector (11) in the plane (10), The photon 
detector can be of any suitable geometry such as in the form of wedges or con- 
centric rings. 

Spatial filtering can also be performed in the plane (10). For example, the 
two basic types of filtering, bandpass and bandstop, can be performed by having 
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a transparent or an opaque region at a desired place in the plane ( 10). Optical 
filters in the form of wedges, annuili or opaque disks are used in the plane ( 10) 
(electrically alterable filters of this type lias recently been delivered to us). 

The light transmitted through the filter-detector (11) is reflected by a parabo- 
loidal mirror segment ( 13) of paraboloid ( 14), placed a focal length away from 
the plane ( 10). The input information is reconstructed at the plane (15), which 
is a focal length away from the paraboloidal mirror (14). 

Figure 4 shows how the system could be housed in an enclosure 15 cm long, 
0 cm high, and 4 cm wide. Experimental performance of the paraboloidal mir- 
ror segment system was evaluated by first monitoring the Fourier transform 
function in the focal plane of the mirror segment using a lle-No laser. The 
following figures show some Fourier transforms obtained with a parabolic 
mirror. As examples of single simple shapes. Figure 5 shows the Fourier 



1 GaAs LASER 

2.. 3 . 4 OFF AXIS PARABOLOIDAL 
MIRROR SEGMENTS 

5 INPUT PLANE 

6 FOURIER TRANSFORM PLANE 

7 RECONSTRUCTION PLANE 


Figure 4. OPDIC Computer 



FOURIER TRANSFORM OF A FOURIER TRANSFORM OF A FOURIER TRANSFORM OF A 
CIRCULAR APERTURE RECTANGULAR APERTURt TRIANGLE 



THE OBJECT TRANSPARENCY FOURIER TRANSFORM (d) ENLARGED VIEW OF 

CENTRAL PORTION OF (e) 

Figure 5. Examples of Fourier Transforms Generated by Parabolic Mirrors 

transforms of a circular, a rectangular, and a triangular aperture. The well 
known Airy disk can be seen distinctly in the center of the transform of the cir- 
cular aperture. Figure 5 (d through 0 also illustrates an example of multiple 
simple shapes. 

The Fourier transform which is a two dimensional distribution of light can 
be sampled to obtain information for decision making purposes. The sampling 
process is simply measuring the amount of light energy falling within a specific 
area and is normally performed by constructing a set of windows of some suit- 
able shapes. By mechanically or manually changing the position of the window 
and monitoring the amount of light in a particular position, a set of measure- 
ments called the "sample signature" can be obtained. 

To overcome the problems of mechanical and manual positioning of the 
sample windows and to decrease the time normally required to produce a sam- 
pling signature we have designed a solid state silicon p-n junction device. This 
device is being used in Fourier transform plane and converts the two dimen- 
sional light distribution into an analog electrical signal. 
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Figure 6. Fourier Plane Photosensors 


Two types of photosensors have been designed. The first is in the form of 
nineteen concentric rings surrounding a two millimeter circle, and is shown in 
Figure 6(a). The width of effective sensitive area of each ring is 0.016" and the 
separation between the two rings is 0.009". The second detector is in the form 
of forty wedges of 9°, separated by 0.008", and the diameter of the central 
circle was 2 mm. Both detectors are one inch in diameter. Figure 6(b) shows 
the wedge type detector. 

Two simple experiments were performed. In the first experiment a trans- 
parency of the letter ”l T " was placed in the input plane. The resulting Fourier 
transform is shown in Figure 7(b). The output voltages on the ring and wedge 
detectors were sampled sequentially and displayed as a histogram on an oscillo- 
scope. These histograms are shown in Figure 7(c and d). Note that the high 
peaks in the wedge detector output corresponds to the straight side of the "U". 

In the second experiment the cloud cover picture was used as an input trans- 
parency and is shown in Figure 8(a). The resulting Fourier transform and the 
histograms are shown in Figure 8(b, c, and d). The two broad peaks in the 
we ige detector histogram corresponds to the directionality of the clouds in the 
input transparency. 
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Figure 7. Photosensor Outputs for Letter "U" 

F ipu re S shows an example of spatial filtering. The Fourier transform of a 
triangle and its reconstructed image are shown in figure 0 (top). To filter out 
one of the sides of the triangle, a narrow wedge was placed in the Fourier trans- 
form was blocked. The effect of blocking the Fourier component is shown in 
Figure 1) (bottom), where it can be seen that the reconstructed image is without 
one side. The missing side has been filtered, or more correctly, suppressed 
in the Fourier transform plane. 

Figure 10 shows another example of spatial filtering. In this experiment a 
He-Ne gas laser was used as a source of coherent radiation mainlv because the 
resolution of the image converter tube used in the image reconstruction plane 
was very low. A wire grid was chosen as an object (fig. 10a). The Fourier 
transform of this object is shown in figure 10b. For spatial filtering purposes, 
a slit was placed in the Fourier transform plane so that only the vertical 
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Figure 8. Photosensor Outputs for Cloud Cover Input 

structure of the Fourier transform was allowed to pass through the Fourier 
transform plane. The reconstructed filtered image (lip. 10c> is without an\ ver- 
tical information. Figure lOd shows the reconstructed intape without am spatial 
filtering. 

Optical correlation experiments using paraboloidal mirror segments were 
also performed. Figure 11 illustrates the optical correlator. The input trans- 
parency was placed in the input plane. A Fourier transform hologram was 
recorded in complex spatial filter plane. The photographic film was developed 
and placed back in exact 1> the same place it occupied during the exposure. The 
reference beam was blocked and the hologram was illuminated In the signal 
beam only. 

Another off-axis paraboloidal mirror segment was used to Fourier-transform 
the field transmitted by the hologram. The correlation plane, which is located 
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SPATIAL FILTER 


RECONSTUCTED 
INPUT IMAGE 


RECONSTRUCTED 
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Figure 9. Spatial Filtering With Paraboloidal Mirror Segments 
and Gallium Arsenide Laser 


a focal length away from the off-axis mirror segment, displays the 1 convolution 
function geometrical image, and correlation function. The autocorrelation has 
a bright central peak of the autocorrelation function can be seen. Correlation 
experiments using a real time optical input device known as "OTTO" were also 
performed. (This device will be described in a later paper by Dr. Jacobson. ) 

Figure Id shows the conceptual design of a coherent optical system with a 
non-coherent image input. The image of the desired part of the earth impinges 
through a beam splitter onto a noncoherent to coherent conversion (OTTO) 
device. The collimated light beam passes through the OTTO device and after 
being reflected from a paraboloidal mirror segment forms the Fourier trans- 
form of the input image. By using a beam splitter the Fourier transform can be 
electrically sampled using special purpose detectors. By placing electrically 
alterable Fourier transform plane filters in the Fourier transform plane various 
features of the image can be detected, enhanced or suppressed. By placing 
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Figure 10. Spatial Filtering Experiments Using Paraboloidal Mirror Segments, 

(a) Wire grid in a circular aperature the object transparency ), 

(b) Fourier Transform of (a), 

(c) Filtered Reconstructed Image, 

(d) Unfiltered Reconstructed Image 
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Figure 1 1 . Optical Correlator Using Paraboloidal Mirror Segments 



Figure 12. Optical Correlation Experiment 
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Figure 13. Conceptual Design of a Coherent Optical System 
With a Non -coherent Image Input 

complex spatial filters in llus plane correlation between two imnr.es can be per 
formed. Suitable sensing devices in the recop.-i meted imar.e plane can provide 
an electrical signal as the output ol the s\stem. 

In conclusion, the results obtained so far indicate that a low power flipped, 
compact coherent optical data processing svstem suitable for on board space 
craft use mil tie constructed. 

AC 'KNOW l I t H , M I N I S 

I'he author wishes to thank Mr. Pn\id II. Schaefer lor Ins eneourar.ement , 
inspiration and support. I'he author was a post doctoral fellow of the National 
Academe of Sciences l .S. \. 

1 .7 


REFERENCES 

1. Abbe, E. Archiv fur Mikroskopischon Anatomie, Vol. 9, p. 413, 1873 

2. Porter, A. B. Philosophical Magazine, Vol. 11, p. 154, 1906 

3. Zernike , F. A. Tech. Phys. , 16, p. 454, 1935 

4. Gabor, D. Nature, 161, p. 777, 1948; also, Proc. Roy. Soc. A 197:454, 

1949 and Proc. Roy. Soc. B64, 449, 1951 

5. Elias, P., et al. , J. Opt. Soc. Am. , 42, 127, 1952 

6. Elias, P. J. Opt. Soc. Am. , 43, p. 229, 1953 

7. Croce, P. and Marechal, A. , Compt. Rend. 237, 706 (1953) 

S. O’Neill, E. L. IRE Trans, of the Professional Group on Information Theory, 
IT-2, No. 2, p. 56, 1956 

9. Cutrona, L. J. , et al. , IRE Trans. Auto. Control, AC-4, No. 2, p. 137, 1959 

10. Van I-Ieerden, P. J. , Appl. Opt. 2, 287 (1963) 

11. Slebodin, L. Proc. I. E. E. E., Vol. 51, p. 1782, 1963 

12. Vander Lugt, A. IEEE Trans. Information Theory, vol. IT -10, p. 139, 1964 

13. Stroke, G.W. An Introduction to Coherent Optics and Holography (Academic 
Press, New York, 1966) 

14. Lohmann, A. W. , et al. , Phys. letters, 25A, p. 570, 1967 


158 



PARALLEL IMAGE LOGICAL OPERATIONS 
USING CROSS -CORRELATION 


Dr. James P. Strong, III 

Goddard Space Flight Center, Greenbelt, Maryland 
INTRODUCTION 

The Goddard Space Flight Center program to develop parallel image proc- 
essing methods has as a basic tenet the concept of an image as input, informa- 
tion as output. One of the most basic types of information contained in an 
image is the number of objects in the image. In a binary or two-level image, 
this amounts to the number of connected areas in the image. Algorithms for 
such counting are known. These algorithms require a succession of logical 
operations on images. This paper will present methods of performing such 
operations in a parallel manner using non-coherent optical techniques. 

LEVIALDI'S ALGORITHM FOR COUNTING 

A parallel algorithm for counting areas in a binary image has been devel- 
oped by S. Levialdi. ^ This algorithm is an iterative operation where the white 
areas are shrunken by incremental amounts until they become single points. 

At each shrinking iteration, isolated points are detected and counted. This 
algorithm is illustrated in Figure 1. For the input image shown, it takes 16 
shrinking steps to count the areas. The algorithm considers an image as a 
matrix array and an isolated point is defined as a white point whose nearest 
neighbors (adjacent and diagonally) are all black. In the input image, there 
are seven connected areas, three of which are isolated points. In the algorithm, 
these are counted and the shrinking operation is then applied forming the first 
iteration. The shrinking and counting operation is continued until all areas have 
disappeared. 

The necessary requirement of Levialdi 's algorithm is that in shrinking 
areas any area will eventually be shrunken to a single isolated point so that 
there is only one isolated point corresponding to any connected area. The 
shrinking function which transforms the image at each iteration of the algo- 
rithm into the image at the next iteration is shown in Figure 2. Here, image A 
is transformed into image B . Each point in image B is a function of a 2 x 2 
element neighborhood of points in image A. The equation of a typical point b^ 
is given by: 
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In order to count the isolated points at each iteration of the algorithm, an image 
containing only the isolated points in each iteration must he obtained. This 
image, (', can be obtained from an image. It. at a given iteration according to 
the equation: 
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Figure). Shrinking Steps of Leviadi 's Algorithm . Input image is at upper left- 
hand corner and last isolated point is in lower right-hand corner. 
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Figure 2. 


Function for Shrinking the Image at One Iteration Into the Image at the 
Next Iteration 


where a. is a typical point in image C. The remainder of this paper concerns 
optical techniques for performing binary image operations which may be used 
to generate the images defined by Equations 1 and 2. 

OPTICAL TECHNIQUES FOR BINARY OPERATIONS 

The addition of two images is one of the most straight-forward optical 
techniques available. One simply superimposes two projected images on a 
screen. Addition of two binary images can be used to perform either the "or" 
operation or the "and" operation. Assume that an input consists of two binary 
images where white has brightness "1" and black has brightness "0". If the 
two images are superimposed, a new image of three gray levels is formed, the 
levels being 0, 1, and 2. "Thresholding" the new image such that all areas 
greater than 1/2 become white and those less than 1/2 become black gives the 
"or" of the two images. If the threshold is set at 1-1/2, the result is the "and" 
of the two images. 

Equation 1 defined each point in a new image as a function of a neighbor- 
hood of points in another image. Corresponding to Equation 1, the whole 
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image can be obtained as a function of shifted versions of another image by 
the equation 

B = (AAA , 0 ) V (AAA, , ) V (AAA„ , ) V (A 0 , AA , Q ) (3) 

Using this nomenclature, "A" is the unshifted image and A xv is image A shifted 
by x elements to the right and y elements up (-x and -y would indicate left and 
down, respectively). Since image 13 is defined as a series of ''and" and "or" 
operations, a series of optical addition, thresholding and storing operations 
can generate image 13 from shifted versions of image A. In order to see how 
shifted versions of an image might be obtained, consider an image formed on 
the back plane of a pinhole camera. If the pinhole is moved up or down relative 
to an initial position, the image (though inverted) will move up or down relative 
to an initial position. If another pinhole were punched in the front of the camera, 
two images would be superimposed on the back plane, one shifted relative to 
the other, corresponding to the positions of the two pinholes. Thus, to add two 
shifted versions of an image, two pinholes can be punched in the front of the 
camera corresponding to the desired shifts. If the image in front of the pinhole 
were binary and two pinholes were used, the image on the back plane of the 
camera could be "thresholded" as described previously, creating the "and" 
or "or" of two shifted versions of the input image. To generate the image de- 
fined by Equation 3, four pinholes arranged in a 2x2 array would be needed, 
generating images A, A 0 , , A , () , and A,, . The pinholes would be capable of 
being opened or closed. In performing the operation, two pinholes would be 
opened at a time corresponding to the four "anded" pairs of images. At each 
step the superimposed images on the back plane would be "anded" by threshold- 
ing at 1-1/2 and the resulting image would be stored. Then the four stored 
images would be "ored" by superimposing all four images and thresholding the 
resulting image at 1/2. 

This operation requires several steps. A slightly different technique will 
perform this operation in one step. Equation 3 can be converted to the following 
threshold logic equation^: 

B = 1 where (3 A + 2A Q , + 2A J0 + A 1 , ) > 3, 

13 = 0 otherwise. (4) 

which says that image B is 1 (white) at points where the summation of shifted 
images is greater than 3 and is 0 (black) where the summation is three or less. 
This equation may be easily verified using a truth table. The summation of 
shifted images within the parenthesis can be obtained again using the pinhole 
camera technique. The same 4 pinholes as described previously are used. The 
two pinholes generating A 0 , and A, 0 are attenuated such that 2/3 of the light gets 
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through. The pinhole for A u is attenuated such that 1/3 gets through and the 
shifted pinhole for "A" is not attenuated. This corresponds to the coefficients 
of 3 for "A", 2 for A 10 and A Q1 and 1 for A n . If the image in the back plane 
of the "multiple pinhole" camera is "thresholded" such that all areas greater 
than 3 become white and all areas 3 or less become black, one has generated 
image B from image A in a single step. 

The summing of shifted versions of an image using the "multiple pinhole" 
camera is, in t "ect, taking the mathematical cross -correlation between that 
image and the "weighted" pinhole pattern. 3 The image on the back plane of the 
camera is the cross-correlation function. Thus, the "multiple pinhole" camera 
is an image cross-correlator. 

A laboratory experiment using the "multiple pinhole" camera was set up 
to perform the operation in Equation 4. In this experiment, however, for 
illumination purposes, the positions of the input image and pinholes were re- 
versed. That is, in place of the input image was an array of pinholes and in 
place of the pinholes was a transparency of the binary image. The cross- 
correlation function is the same in either case. This cross-correlator is dia- 
grammed in Figure 3. The pinhole array is image A and the binary image is 
image B. The scale indicated for image B relative to image A produces the 
correct increments of image shifting. The pinhole array used in the experiment 
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Figure 3. Cross-Correlator for Shifting Images. Sum of shifted images appears in 
cross-correlation plane. 
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was actually the array of square aperatures shown in Figure 4. This allowed 
the simple shading technique shown to he used to give accurate relative 
attenuation. 

An example of the imagery obtained in this experiment is shown in Figure 5. 
The input image (upper left) is cross-correlated with the array in Figure 4 pro- 
ducing the cross -cor relation function in the upper center of Figure 5. Below 
this function are the numerical values associated with the gray levels. "Thresh- 
olding" the cross-correlation function at ."-1 2 would produce the desired 
shrunken image (upper right). A device for doing this thresholding over an 
entire image does not presently exist. However, a device that has been devel- 
oped by Hughes Research Laboratories designed to spatially modulate a laser 
beam (called the OTTO device) has a steep gamma curve and, therefore, an 
attempt was made to use this device as an image thresholder. 

The experimental setup is shown in Figure (!. Here, the multiple pinhole 
camera system to the left formed the cross-correlation function on an image 
intensifier tube. The output id' the tube was then focused onto the surface of 
the OTTO device which was read out with the laser beam. 1’he gain of the 
image intensifier tube was adjusted until thresholding occurred at about 2-1 2. 
Figure 7 shows the results. The image on the top left was cross-correlated 
with the pinhole array in Figure 4 producing the cross-correlation function in 
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Figure 4. "Attenuated Pinhole Array" for Shrinking 
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the top center. I'he tlesi red thresholded result is at the t < > right. l'he out | Hit 
of the OTTO device (whieli is :i negative) is seen in the lower part of Figure 7. 
This was basically a feasibility experiment and no attempt was made to assure 
uniform sensitivity or illumination over the surfaces of either the image intensi- 
fier or the OTTO device. 1'lie OTTO device was an interim device loaned by 
Hughes for e\ aluation. 

A MOHK rOMPI.FX OPFKA HON 

Fite results shown in Figure 7 show that the OTTO device can do some 
"thresholding' 1 between the levels of and 1 but not \er\ effectively. However, 
the device will "threshold” effectively between the levels of n and 1. That is. 
it can be made to "saturate” at the lowest incremental brightness level above 
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Figure 5. Demonstration of Summing of Shifted Images Using Multiple Pinhole or 

Cross -Correlation Technique. Input image is at upper left. Summation of 
shifted images is at upper center. Results of thresholding is at upper right. 
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FUNCTION 


zero. Using this property, the following experiment was performed which pro- 
duced the image of isolated points necessary in Levialdi's counting algorithm. 

Corresponding to Equation 2, an image, C, containing only the isolated 
points in another image, B, can be obtained from shifted versions of image B 
using the following function: 

Bq,i AB-ii AB 


where B is the negative of image B. In the experiment, it was more convenient 
to use C which can be written as: 


C=B 1 > _i v B 0 i . 1 v B_ 1 ) _i v Bi ) o v B_i j 0 v B 1 i i v 


B<,.i VB -U VB 
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Figure Thresholding of Cross-Correlation 

Function Between Levels of 3 and 4 
Using OTTO device 


This equation when converted to threshold logic becomes: 

f 1 w here t B , , + B ( , , + B , , + B, 0 * B ,,, ♦ B, , +B, U + B ^0 


c 0 elsewhere 


(">) 


The first step in the experiment was to sum the shifted versions of image B 
within the parenthesis in Equation >. Fhis was done by cross-correlating 
image B with the pinhole array shown in Figure S. I’he experimental setup is 
illustrated in Figure n where the pinhole array illuminated the transparency of 
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Figure 8. "Pinhole Array" for Generating Image of Isolated Points 
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Figure 9. Optical System for Generating Image of Isolated Points 
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image B forming the cross-correlation function or summation of shifted images 
on the input of an image intensifier tube. A negative transparency of image B 
was projected onto the image intensifier input so that it added to the summation 
of shifted images. This corresponds to the B term in Equation 5. The output 
of the image intensifier tube was focused onto the input of the OT TO device. 
Here the total summation of images was thresholded at about 1 2 and, since the 
OTTO device produces a negative, the laser read out image ('. Figures 10 and 
11 show results. Figure 10a shows an input image with one isolated point. In 
10b there is shown the cross-correlation function between the input image and 
the pinhole array, and 10c shows the isolated point in the output of the )TTO 
device. Figure 11 shows the same results when the input image was the input 
image of Figure 1 with 3 isolated points. 



Figure 10. Detection of a Single Isolated Point, a) Input Image, (b) Cross- 
Correlation Function Between Input Image and "Pinhole Array" of 
Figure 8, (c) Isolated Point Output of OTTO Device 



Figure 11 . Detection of Multiple Isolated Points: ia) Input Image, (b) Cross- 
Correlation Function Between Input Image and "Pinhole Array" of 
Figure 8, (c) Isolated Points in Output of OTTO Device 






CONCLUSIONS 


This work has described very simple optical techniques for performing 
complex binary operations on images. In cases where binary equations can 
be converted to single threshold logic equations, a single cross-correlation 
operation can take the place of multiple image shifting, "anding", "oi'ing" and 
storage operations. 
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DIGITAL HOLOGRAPHIC LOGIC 


IC. Preston, Jr. 

The Perkin-Elmer Corporation, Norwalk, Connecticut 06856 


INTRODUCTION 


N73- 18696 


Optical computers ordinarily operate in an analog mode where signals 
introduced into the computer take on continuous values. The computing rate 
achievable in such computers is given by 


R 


C 


erjP 

hiJS/N) 2 


( 1 ) 


where e is the optical transfer efficiency, r? is the quantum efficiency of the 
output detector, P is the optical input power, h is Planck’s constant, v is the 
optical frequency, and (S/N) is the desired rms output signal to noise ratio. 

For typical values such as e = 0.5, ij = 0.1, P - 1 .0 watt, v = 10 15 Hz, S/N - 1 0, R 

1 i C 

is ofthe oi'der of lCr multiplications per second. Accuracy, however, is low. In 
ordinary optical systems the accuracy level is 5 bits, but with extreme care in both 
optical design and lens element fabrication, 7 to 8 bit accuracy maybe achieved (1). 

Ordinarily the dynamic range in the optical analog computer is 40 db (13- 
bits). Again, with extreme care in design, fabrication and assembly 60 db 
(20-bits) may be reached. In comparison with the electronic digital computer, 
where in many cases 128 bits accuracy is available for double precision arith- 
metic, the optical analog computer is far from competitive. 

However, it is possible for optical computers to achieve high accuracy by 
operating in a digital mode using signal levels which are restricted to the binary 
values (0 and 1). The purpose of this paper is to describe experiments con- 
ducted at Perkin-Elmer demonstrating the feasibility of an optical digital com- 
puter. The program of this computer is entered holographically. This would 
be done under control of an electronic digital control unit (see Figure 1). The 
digital control unit would store the program and would also act as a storage 
medium for binary number arrays. These arrays would foi*m the input to the 
optical computer and similar arrays would be retrieved from the optical com- 
puter when calculations were completed. 

In order to satisfactorily combine the relatively slow electronic digital 
control unit with the optical computer, it is necessary that the optical computer 
iterate through many stages of a calculation before either requesting more data 
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Figure 1 . Block schematic showing connections between the electronic control 
computer and the parallel optical computer for use in performing 
digital logic holographically. 

at its input or transferring results from its output. The optical computer would 
thus be built from a concatenation of blocks of the type shown in Figure 2. The 
block shown in this figure consists of a combination of optical-to-optical conver- 
sion devices (OTTO's) interspersed with computing elements consisting of lens 
pairs and holograms. 

This paper discusses the characteristics of the block shown in Figure 2. 
First the holographic operation is reviewed from the Fourier transform view- 
point. The paper then describes the formation of holograms for use in 
performing digital logic, illustrates the operation of the computer with an 
experiment in which the binary identity function is calculated, then discusses 
devices for achieving real-time performance, and describes an application in 
pattern recognition using neighborhood logic. 

HOLOGRAPHY 

Figure 3 shows a schematic representation of the first stage of the block 
shown in Figure 2. This block makes use of the lens in taking the Fourier 
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Figure 2, Schematic arrangement of the basic building block of the holographic 

logic computer. Two optical to optical transducers, one hologram, and 
two Fourier transform lenses are required. 


transform. The x ( plane is the input plane, L, 7 is the transform lens, and the x, 
plane is the Fourier transform plane. These planes are separated by two focal 
lengths. The lens action of the transform lens is given in the below equation (2). 




-ju» x X, 

E(x,)= / E(x,)e dx. 


( 2 ) 


where E(x,) is the electric field distribution in the input plane, E(x,) is the elec- 
tric field distribution in the output plane and co K = 2i rx 0 /AF where A is the optical 
wavelength and F is the focal length of L, 2 . 

Assume that the x, plane contains a spatial light modulating device whose 
light modulating elements are discrete and regular in position. One such ele- 
ment is located at x, ( . Light emanating from this point may be considered to be 
a positional delta function which is transformed into a plane wave in the x, plane 
as follows. 

r -jco x x, -jw x x a 

E(x 2 ) == J<5(x | --x a )o dXj = e (3) 
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E fw )= | E 

V x 2 J J 

w = 2irx,/XF 
x 2 * 




8 ( x r x a) 

Figure 3. Analytical relationship between a discrete input at x g in the input plane 
and the associated plane wave in the Fourier plane created by the trans- 
form lens L 1 2 . 

When the second lens of the block is introduced, the block is complete and 
the configuration is as shown in Figure 4. The lens L 23 takes a Fourier trans- 
form with the result appearing in the x 3 plane. The reader should note that, 
since lenses in optical computers take successive Fourier transforms, the 
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Figure 4. Analytical relationship between the discrete input at x a in the input 
plane and its ima^e at -x in the output plane when two successive 
Fourier transforms are taken using lenses L 12 and l_ 23 . 


inverse of the input function appears at the output, namely, a delta function not 
at x a but at -x a . This is expressed in the following equations 


E(x 3 ) 


= / E(x 2 )e ' 




--K (x 3 +x a } 



= 5(x 3 +x a ) 

where co x = 27rx 3 /XF and u/ = 27rx 2 /XF. 

In order to form a hologram and illustrate its operation upon an input delta 
function at x a consider the configuration diagrammed in Figure 5. Two discrete 
inputs are provided: one at x a ; one at x t . Lens L 12 carries out the Fourier 
transform of these two delta functions to produce the electric field distribution 

" j “* X a " jt Vr 

E(x 2 )-e +e ( 5 ) 



T 




Figure 5. Equations describing the generation of a hologram in the Fourier plane 
using a discrete input at x a and a reference at x f in the input plane. 

An energy detector placed in the x, plane records the hologram H given as 
follows 

H ar = E(x 2 )E*(x 2 ) = 2(1+ cos co x (x r -x a )l (6) 

Note that this hologram consists of a bias term plus a cosinusoidal term whose 
spatial frequency is proportional to the quantity (x r - x a ). 

When this hologram is placed in the optical computer and a discrete input 
occurs at x , the result is as shown in Figure 6. Once again the lens L p 
generates a plane wave in the x 2 plane in which the hologram H is located. The 
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Figure 6. Schematic showing the results when a discrete input at x g is used in 

conjunction with a hologram H gr in the Fourier plane to produce three 
discrete outputs in the output plane. 


plane wave interacts with H gr in such a way that three plane waves are created 
which are transformed by L 23 into three delta functions according to the below 
equation 

" iw x ( VV - jt Va “K x 2 

e e dx„ 


ff J w x< x r x a> 

=J [ 2+e 

■A 


+ e 


K (x 3 +x a> -j^x (x 3 +x r) “^x (x 3 'V 2x a ) 

2e + e + e dx„ 


(7) 


- 25(x 3 + x a ) + 5 (x 3 + x r ) + 5 (x 3 - x r + 2x a ) 


The reader familiar with matched filter theory will recognize that the delta func- 
tion at -x r is the peak of the correlation function generated by H ar acting as a 
matched filter in the frequency domain (3) . 

Another way of diagramming the action of H r is shown in the left hand por- 
tion of Figure 7. This simplified diagram indicates only the input plane (hori- 
zontal line at top) and the output plane (horizontal line at bottom) . The action of 
the entire lens system in combination with H ar is shown merely by the downward 
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Figure 7. Abbreviated schematics showing (left hand drawing) the result given in 


Figure 6 and (right hand drawing) result when using two discrete inputs 
in the input plane at the points x g and x fa . 

directed arrow. On the line representing the output plane the numerics indicate 
the electric field strength of three delta functions. Note that the electric field 
strength at -x is twice that of the other two delta functions. 

HOLOGRAPHIC LOGIC 

In performing holographic digital logic it is important to understand how the 
hologram interacts with light emanating from a multiplicity of discrete inputs in 
the x x plane. The right hand portion of Figure 7 shows what occurs when a sec- 
ond input is added at x b . New delta functions appear in the output plane separated 
by the distance (x b ~x a ) from the other delta functions. They are of identical 
electrical field strength and of equal phase if we assume that the input at x b has 
strength and phase equal to that at x . 

In order to perform digital logic holographically it is necessary to combine 
at one point in the output plane energy emanating from two discrete inputs in the 
Xj plane. This may be done by using a somewhat more complex hologram con- 
structed in the same manner as shown in Figure 5 but with discrete inputs at 
both x a and at x b simultaneously. The electric field strength in the x, plane is 
given by the equation 

■ -j“V x a ~KV -j w .N x r 

E(x 2 ) = e + e + e (8) 
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Figure 8. Abbreviated schematics showing the results of using the hologram H in 


the Fourier plane with a single discrete input at x g (left hand drawing) 
with two discrete inputs at the points x g and x b (right hand drawing). 

and the hologram H abr resulting from exposing an energy detector to this field is 
given by 

H abr = E ( X 2 ) E *( X 2) 

= 3 + 2 [ cos co x (x r - X a ) + cos co x (x r - x b ) + cos o? x (x b - x a )] 

As can be seen there are three cosinusoidal terms. If we assume, as is usually 
the case, that the reference input at x r is considerably removed from the optical 
axis, then two of the cosinusoidal terms represent high spatial frequencies while 
the other has a relatively low spatial frequency related to the difference in posi- 
tion between x b and x a ). 

When hologram H abr is introduced in the x, plane and an input at x a is present 
in the x 1 plane, the resultant output electric field distribution is shown in the left 
hand side of Figure 8. The reader should compare this result with that shown 
in the left hand side of Figure 7 which relates to the same input configuration 
when using hologram H ar . 

When using hologram H abr a single input in the x x plane produces a holo- 
graphic image of the input plane configuration for which H abr is the hologram. 
Thus not only does an output delta function appear at -x but there is another 
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Figure 9. Electric field strength in the output plane at the point -x produced 
using each of the four binary combinations of inputs at x g and x b . 


delta function at a distance (x b -x a ) from it. Holographically speaking this is 
the "reconstruction" of the input configuration for which H br is the hologram (4). 

The final and most important result is shown on the right hand side of Fig- 
ure 8. Here the configuration of inputs in the x } plane is identical to that from 
which the hologram H abr is made. When this is the case, the electric field 
strength at the point -x in the output plane is the coherent summation of the 
electric fields at x and x b in the input plane. Speaking from the point of view of 
matched filter theory, the electric field strength in the vicinity of -x r is the 
correlation function of the input configuration whereas that in the vicinity of the 
conjugate region at [x - (x. +x )] is the convolution function. Other inputs 
appear near the optical axis. These have the highest electric field and occur 
where the input configuration is imaged by the combined action of L p and L, 3 . 

In using holograms such as H b for performing digital holographic logic 
only the delta functions near - x is of importance. Therefore in the discussion 
which follows only this region of the output plane is treated. 

THE BINARY IDENTITY FUNCTION 

In order to perform digital logic using H abr it is necessary to elect a method 
for introducing the binary values of the input variables. The most straight- 
forward approach is to use zero phase for the binary 0 and the opposite phase 
(180°) for the binary 1 , The resultant field strength in the output plane at the 
point -x r for this selection of the binary variables at x ;i and x b is tabulated in 
Figure 9. In all cases it is assumed that the electric field strengths at x and 
x b are identical. As can be seen, when the phase of the electric field at x and 
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x ( , are the same, the output field strength at x r is finite. When the phases are 
opposite, the output at-x f is zero. Optically speaking, finite and zero outputs 
correspond to constructive and destructive interference, respectively. Logically 
speaking, the tabulation shown on Figure 9 may be considered to be a truth table 
for the identity function. 

In order to completely appreciate the appearance of the output plane, reter 
to Figure 10 which shows in the two left hand columns the states of the input 
variables and in the three right hand columns the complete correlation function 
configuration in the vicinity of the point x f . Not only is the identity function 
(center column) shown but also the light distribution in its immediate vicinity 
for all four possible pairs of values of the input variables. 


EXPERIMENTAL CONFIRMATION 

An experiment was recently carried out in the laboratory where such a 
computation was performed using a hologram of the type given by H , . Specifi- 
cally this hologram was created using two discrete inputs consisting of 50 /lmii 
diameter pinholes with a 100pm separation. An input light modulator was then 
created by vapor deposition which is shown in Figure 11. This light modulator 



Figure 10. Diagram of the autocorrelation functions produced in the vicinity 
of the point x f in the output plane for the four possible binary 
combinations of inputs at points x i and x ( . 
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Figure 1 1 . Photomicrograph of a two dimensional spatial light modulator having 
four phase modulating regions (encircled) whose action is to shift the 
phase of transmitted light by 180 degrees with respect to light passing 
through adjacent regions. 


had its elements on a grid spacing of 100pm. Each element has 50pm in diam- 
eter. Four elements were created which shifted the phase of the transmitted 
light by 1K0° with respect to light passing through the background. These light 
modulating elements were arranged so that they would correspond to the input 
variable combinations given in Figure 10. 

Figure 12 shows the resultant light intensity distribution in the output plane 
when the light modulator was placed in the input plane. There is a background 
of in-phase illumination. Regions 50pm in diameter showing both constructive 
and destructive interference may also be observed, tn the central column the 
expected binary identification function is displaved. The "bright” output expected 
for the fourth value of the identity function is surrounded by a dark halo due to 
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Figure 12. Photomicrograph of the output light intensity at a column of points 
(dashed lines) at x r obtained when performing the binary identity 
function using digital holographic logic. 


the fact that the high spatial frequencies corresponding to the border of the light 
modulating elements were not passed by the optical computer lens system. An 
improvement in performance was suggested and demonstrated bv Dr. Ronald 
Grosso of Perkin- Elmer with the results shown in Figure 13. In this case the 
l's complement of the binary identity function, namely, the Exclusive OR. was 
created. This was done by translating H )|ir bv one half cycle of the spatial fre- 
quency corresponding to the final term in equation (9). This has the effect of 
producing a relative phase shift of 180° between the output generated from the 
input at wi h respect to that generated from the input at x h . Thus all regions 
of constructive interference are replaced by regions of destructive interference 
and vice veisa. The only regions of constructive interference exist where the 
values of th nput variables are unequal in phase. As can be seen the * gnal to 
noise ratio is considerably enhanced. Thus is would appear that an inverting 
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Figure 13. Photomicrograph of the light intensity in the output plane at point x ( 
(dashed lines) obtained when performing the Exclusive OR function 
using digital holographic logic. 


logic corresponding to the I's complement is more practical to use in optical 
computers performing digital holographi logic than the direct logic whose 
results are shown in Figure 12. 


OPTICAL TO OPTICAL CONVERSION 

In order to make coherent optical computers using digital holographic logic 
a practical reality it is necessary to obtain OTTO’s for transducing the light 
intensity distribution in the output plane into appropriate input signals for the 
next block in the computer chain. One possible OTTO is shown schematically 
in Figure 14. This device is the PMLM (Photosensitive Membrane Light 
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Figure 14. Isometric drawing showing the basic configuration of the photo 
sensitive membrane light modulator (input surface below; output 
surface above). 


Modulator) originally described by Reiznian (5). The PMLM is made from a 
thin silicon substrate using integrated circuit technology. When n-type silicon 
is used, a heavy n+ diffusion is made on the input surface to act as a transparent 
electrode. On the opposite surface p+ diffusions are made having the size and 
spacing of the required input array of phase modulating elements. These dif- 
fusions form an array of pn junction diodes. Overlaving the diodes is a resistive 
layer consisting of a semi-insulating material approximately 1 m thick. This 
layer is interrupted with circulm perforations registered with the p regions of 
the diode array. 

Three more layers complete the device. First there is a collecting elec- 
trode on top of the semi-insulating layer (but not extending over the circular 
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perforations), next is a polymer membrane which is stretched tightly across the 
perforations, and finally, a layer of metalization which makes the polymer mem- 
brane reflective. 

The PMLM is energized by a power supply poled so as to back bias all the 
diodes in the array. This places the positive terminal of the power supply on 
the transparent electrode and the negative terminal is common to both the col- 
lecting electrode and the membrane metalization. When the input surface of the 
PMLM is not illuminated, all diodes aie back biased and, ideally, no current 
flows through the semi-insulating layer. This means that the entire potential of 
the power supply appears across the pn junctions leaving all p regions at the 
potential of the collecting electrode (and the membrane metalization). When 
light strikes locally in the vicinity of one diode, this diode conducts and the p 
region now becomes more positive. This introduces a potential difference 
between the p region and the membrane metalization. Oue to electrostatic 
forces the membrane locally deflects. Phase modulation of reflected light 
results from this deflection. 

The membrane surface of the PMLM is the output surface and it is coher- 
entlv illuminated from an optical source. By adjusting the optical computer 
parameters properly, phase modulation due to deformations of the membrane 
elements may be made to correspond to the 0° and 180° points necessary for 
doing binary logic holographically. Interferograms showing the operation of 
two elements of an early PMLM are ohown in Figure 15. Each interferogram 
is of the same portion of the PMLM. Two circular elements are shown which 
are approximated 40pm in diameter on approximately 100pm centers. From 



Figure 15. Interferomicrograms showing deflections of the output surface of a 
photo sensitive membrane light modulator (left to right): power off; 
power on, no illumination; power on and illuminated. 
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left to right in Figure 15 the first interferogram shows the membrane elements 
undeflected when no power is applied to the PMLM. With the power supply 
adjusted to 70 volts , both elements deflect due to spurious leakage current 
through the corresponding diodes. The deflection produces a phase modulation 
of the order of 180°. This phase level must therefore be utilized as the zero 
degree phase reference. When the input surface of the PMLM is illuminated, a 
further deflection takes place as is shown in the right hand interferogram. With 
proper design a further increase of 180° may be achieved. Thus the actual 
operating states of the membrane elements in question would be 180° to 360° 
rather than from 0° to 180°. 

CONCLUSION 

Although all propositional logic may not be carried out using the identity 
function or the exclusive OR, it is possible to expand the above techniques into 
a complete logical set by incorporating the logical OR function. To accomplish 
this one must use a three -state logic employing three input elements for each 
two-input function. One element is always at zero degrees of phase. The holo- 
gram used is made with three input elements at 0°, 120°, and 240°. The input/ 
output truth table is as follows: 


Input A Input B Output 


0° 

0° 

Zero 

0° 

180° 

Finite 

180° 

0° 

Finite 

180° 

180° 

Finite 


The initial entry in the truth table causes the hologram to create a coherent 
summation in the output plane of three equal vectors of phase 0°, 120°, and 240°, 
respectively. This produces a null condition (destructive interference) and 
therefore a 0 output. In all other cases, where 180° is added to either or both 
of the inputs, the vector summation in the output plane is no longer null and a 
finite output occurs. 

Using this simple modification of the original two-state logic described 
above, all propositional logic may be performed using the appropriate hologram. 
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WORKSHOP REPORTS 


WORKSHOP NO. 1 

Mr. Schaefer: Arnold, would you mind telling us, what are desirable 
image processing tasks to perform onboard Earth Observation Spacecraft and, 
in contrast, what are the desirable tasks that lend themselves to image proc- 
essing on the ground ? 

Mr. Shulman: Workshop No. 1 could not answer that question and many 
others that we raised. 

In fact, I think we have gotten ourselves more problems than we have 
solved. 

In talking with Mike, and just overhearing the discussion here, it seems 
that it is unanimous that we don't know what type problem we have, and what 
we are trying to solve. 

As far as airborne processing goes, we don't know what answers we are 
looking for. 

Right now, because of the user community requesting so many varied and 
different types of information, it only seems reasonable at this point in time 
to try to bring down as much of the information as possible, and process it 
with a hybrid parallel type processing on the ground to extract the specific 
types of information that the user community wants. 

However, in regard to that, we strongly feel the need for some sort of 
standardization group to be formed to come up with a catalog, if you will, that 
presents the types of information that can be processed, or what the user com- 
munity might expect, say, in five years, as a type of data processing that they 
should design their experiments around. 

So that five years from now, the user community has been educated to 
feel, that from this catalog, — if I design my experiments this way or that, I 
will be able to get good data and it will limit the type of airborne computations 
that we may need to provide in the air. 

We felt there might be a problem with present technology with regard to the 
bandwidth requirements for transmitting data to the ground. We could have 
addressed ourselves to bandwidth requirements with laser communications, 
which might be able to solve this type problem. If we do transmit all data to the 
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ground to be processed, it should be put in a format which would permit easy 
processing, which means some sort of memory device which doesn't ruin 
the information. Film right now is probably one of our better mediums for 
storing large amounts of data but is not the complete answer as it exists now, 
because there are considerable problems caused by the way it must be handled 
and the way it stores information. This is another area that may require 
looking into. 

Once we have decided what type of information the user community wants, 
it then would become a relatively simple job to design airborne specialized 
computers to solve specific requirements, provided we could limit the require- 
ments to these specific ones. 

This gets back to the need for some sort of standardization committee to 
be formed to provide some guidance to the user community, as well as the 
designers, so that we know what to design for. 

I believe that is as reasonably accurate results as I can give you from our 
workshop. 

Mr. Schaefer: Can I sum it up? Do I read you right that— 

Mr. Shulman: Mine was a summary. 

Mr. Schaefer: —they came up forescore for a committee. 

Mr. Shulman: Right. 

Mr. Schaefer: Very good. It sounds like you had a very worthwhile time 
to me, even though you didn't answer the questions. I bet none of us did. 

WORKSHOP NO. 2 

Norman, how can image processing units aboard low altitude orbiters, 
those aboard synchronous orbiters, and those on the ground cooperatively 
interact ? 

Dr. MacLeod: Well, we viewed these questions as more or less potential 
users. We are talking about a future system in the 1980s, 1990s time frame, 
and I think we all agreed that it would be very nice to have a synchronous satel- 
lite that would beam every kind of desirable information into a farmer's TV. 
This was an ultimate system. Then we begin to back off from there. One has 
to consider the conflicting uses of many potential users. 
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We also considered that by that time people would know what they wanted 
they would be, perhaps, used to acquiring and using remote sensing data. 

Now, from that, there were some comments concerning the type of ground 
station one might have. We felt that an experimental spacecraft system would 
always be accompanying the operational spacecraft. We felt that simple sys- 
tems like the APT, which is essentially a farmer's TV in the year 1990, might 
be a useful kind of thing for operational needs where a map is coming down 
directly, very much like a weather map. Such map producers would employ 
onboard processing in either synchronous or low orbital spacecraft. We how- 
ever felt that complex ground systems dealing with high data rates would be 
involved with experimental spacecraft. 

There was some discussion about whether or not it was useful to relay 
very small amounts of information down, whether characterizers or classi- 
fiers onboard the spacecraft could usefully send down simple indices of 
various states of surface information or meteorological information. This 
probably would be limited to users that had quite well known needs. 

We also considered the frequency of observation. In some of these satel- 
lites we can get a picture down every few minutes from synchronous altitudes, 
but many of our users don't need this kind of frequent information. This has 
an effect on the interaction or where you are going to process information. 

One of the obvious places where the systems can interact, of course, is 
between the spacecraft that would be, perhaps, in a polar orbit, and a geo- 
stationary spacecraft essentially acting as a relay system. 

We had a particularly difficult problem, I think, with knowing how much 
we were going to know in the future, and also knowing what kind of sensors 
and computational devices there would be available under such systems. 

We had some discussion on these matters and I don't think we came to any 
particular conclusion about this, except that we felt that the users probably 
would know what questions they had in mind and that some of us, in fact, do 
know already what questions we have in mind. It is a matter of knowing to 
what stage of understanding of the application of that data that we would have 
in the future, which would affect the processing systems in the manners that I 
have just spoken about. 
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The problem is to supply only the required information to the users, rather 
than having great masses of data coming down. In this sense the orbital space- 
craft, such as the ERTS spacecraft, is going to supply a great amount of infor- 
mation for some users that don't need great amounts of information, and not 
enough information for some users that need very frequent observations. And 
this situation, of course, would continue with the multistage kind of analysis 
that is going on, for instance, in forestry. 

We returned several times to the question of multistage sampling and using 
these systems as direct interactive systems, where the altitudinal differences 
in spacecraft and aircraft would be utilized in a multistage sampling system. 

But again the problem is , what are we going to know when the time comes 
to use these systems, and this will in effect assign us what we are going to be 
doing. 

Mr. Schaefer: It sounds most fruitful, very interesting. 

WORKSHOP NO. 3 

Mr. Schaefer: Bill, did you answer your question? Let me read it. 

Mr. Alford: I was going to read it, because that's half of my summary, 
reading the question. 

What Earth Observation tasks are uniquely suited to coherent optical proc- 
essing systems ? To non-coherent systems ? To parallel electronic systems ? 

No one dared admit that there was any one task that was uniquely suited. 
And that is the conclusion. 

Now I will give you a little background. 

We tried— just to give you a little of the frustration we went through— we 
tried to define what we were going to talk about and we decided that, first of 
all, let's look at the information objectives, you know, what are we really 
after? And I guess like everybody else, that quickly ground to a halt because 
no one could define that. 

I guess primarily we have a lot of the technology people and not a lot of the 
end-users here, at. least we didn't have a lot of end-users in our group to 
ansv/er that question. 
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So we backed off and we said, well, let's look at techniques, and maybe 
we can define where those techniques fitted a little better in these three 
categories. 

So we started looking at a lot of techniques, and I will just mention a 
couple. 

We got off on correlation for picture rectification, radiometric correc- 
tions, multispectral classifications, enhancement, etc. You have heard a lot 
of them. And we started looking at some of these and no one recognized any 
of these as being uniquely suited to any of the three systems. 

We think there are probably still some holes left like, in terms of applica- 
tion, like what can you do with coherent optics in teirns of multispectral 
classification ? We feel that maybe all we need are opinions or just comments 
from certain people on some of these questions. 

We sort of concluded that in many cases the application probably has to 
be analyzed in terms of cost tradeoffs and throughput requirements. Again, 
nothing conclusive other than in many cases each situation has its own solu- 
tion. 

I wrote down what Arnold said, and I will say it again. 

Once we decide what the information is that the user community wants, 
we can decide which techniques are going to be applicable. 

Mr. Schaefer: I don't dare to summarize your summary. 

WORKSHOP NO. 4 

Mike, can you tell us, what are the image processing requirements for 
small easily available ground stations ? 

Mr. Michalak: Well, Dave, no, I don't think I can. 

When we started talking about the APT station we very rapidly came to the 
conclusion that the data gathering end of such a ground station would probably 
be prohibitively expensive within the next two or three years, talking on the 
order of a quarter of a million dollars for an auto track antenna alone. 

Now once we kind of did away with that, we started looking around as to 
how could we then reformat data, and how could we put it into such a form 
where an APT antenna might be more worthwhile ? 
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We came around to the general idea of data management as starting out 
with a satellite which perhaps costs you hundreds of millions of dollars. 

From the satellite, such as ERTS, we would beam down through a high 
data rate link to several global regions and in each global region would be a 
central menial task center, menial task being to reformat the data, put in all 
the latitude and longitude, do radiometric correction, that sort of thing. These 
things would run you about $25 million-or-so apiece, perhaps even a little bit 
more with inflation. 

Then from the reformating stations, we go back to a geosynchronous 
orbiter and from the geosynch satellite we would beam down to district offices. 

Now when we start talking about district offices, we started talking about 
the kind of market that earth resources and parallel image processing is going 
to have, and again we decided that we are going to have to develop our own 
market. 

In other words, echoing what I have heard here today, we have to educate 
the user community as to what parallel processing can do, and more specif- 
ically, we have got to go out and demonstrate useful applications of this. 

Now admittedly we don’t know all of the possible applications, but things 
like flood control, area pollution monitoring, fault line detection, various 
things like this are probably good places to start to get a little handle on what 
is happening. 

So I guess basically the problems that we ran into are 

(1) the high data lines, and 

(2) do we need parallel processing, if so, show me, show me what it is 
going to do for me. 
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